-- Logs begin at Tue 2022-08-23 02:46:51 UTC, end at Tue 2022-08-23 04:52:44 UTC. -- Aug 23 04:13:23.239835 dv-cinder-854070-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 23 04:13:23.252595 dv-cinder-854070-1-iscsi devstack@keystone.service[81919]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 23 04:13:23.253806 dv-cinder-854070-1-iscsi devstack@keystone.service[81919]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 23 04:13:23 2022] *** Aug 23 04:13:23.253806 dv-cinder-854070-1-iscsi devstack@keystone.service[81919]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 23 04:13:23.253806 dv-cinder-854070-1-iscsi devstack@keystone.service[81919]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 23 04:13:23.253806 dv-cinder-854070-1-iscsi devstack@keystone.service[81919]: nodename: dv-cinder-854070-1-iscsi Aug 23 04:13:23.253806 dv-cinder-854070-1-iscsi devstack@keystone.service[81919]: machine: x86_64 Aug 23 04:13:23.253806 dv-cinder-854070-1-iscsi devstack@keystone.service[81919]: clock source: unix Aug 23 04:13:23.253806 dv-cinder-854070-1-iscsi devstack@keystone.service[81919]: pcre jit disabled Aug 23 04:13:23.253806 dv-cinder-854070-1-iscsi devstack@keystone.service[81919]: detected number of CPU cores: 4 Aug 23 04:13:23.253806 dv-cinder-854070-1-iscsi devstack@keystone.service[81919]: current working directory: / Aug 23 04:13:23.253806 dv-cinder-854070-1-iscsi devstack@keystone.service[81919]: detected binary path: /usr/bin/uwsgi-core Aug 23 04:13:23.253806 dv-cinder-854070-1-iscsi devstack@keystone.service[81919]: your processes number limit is 47680 Aug 23 04:13:23.253806 dv-cinder-854070-1-iscsi devstack@keystone.service[81919]: your memory page size is 4096 bytes Aug 23 04:13:23.253806 dv-cinder-854070-1-iscsi devstack@keystone.service[81919]: detected max file descriptor number: 2048 Aug 23 04:13:23.253806 dv-cinder-854070-1-iscsi devstack@keystone.service[81919]: lock engine: pthread robust mutexes Aug 23 04:13:23.253806 dv-cinder-854070-1-iscsi devstack@keystone.service[81919]: thunder lock: enabled Aug 23 04:13:23.256781 dv-cinder-854070-1-iscsi devstack@keystone.service[81919]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 23 04:13:23.256781 dv-cinder-854070-1-iscsi devstack@keystone.service[81919]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 23 04:13:23.292980 dv-cinder-854070-1-iscsi devstack@keystone.service[81919]: Python main interpreter initialized at 0x55888d4199c0 Aug 23 04:13:23.292980 dv-cinder-854070-1-iscsi devstack@keystone.service[81919]: python threads support enabled Aug 23 04:13:23.292980 dv-cinder-854070-1-iscsi devstack@keystone.service[81919]: your server socket listen backlog is limited to 100 connections Aug 23 04:13:23.292980 dv-cinder-854070-1-iscsi devstack@keystone.service[81919]: your mercy for graceful operations on workers is 90 seconds Aug 23 04:13:23.293356 dv-cinder-854070-1-iscsi devstack@keystone.service[81919]: mapped 403077 bytes (393 KB) for 2 cores Aug 23 04:13:23.293356 dv-cinder-854070-1-iscsi devstack@keystone.service[81919]: *** Operational MODE: preforking *** Aug 23 04:13:23.293669 dv-cinder-854070-1-iscsi devstack@keystone.service[81919]: *** uWSGI is running in multiple interpreter mode *** Aug 23 04:13:23.293669 dv-cinder-854070-1-iscsi devstack@keystone.service[81919]: spawned uWSGI master process (pid: 81919) Aug 23 04:13:23.293887 dv-cinder-854070-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 23 04:13:23.294941 dv-cinder-854070-1-iscsi devstack@keystone.service[81919]: spawned uWSGI worker 1 (pid: 81933, cores: 1) Aug 23 04:13:23.294941 dv-cinder-854070-1-iscsi devstack@keystone.service[81919]: spawned uWSGI worker 2 (pid: 81934, cores: 1) Aug 23 04:13:23.294941 dv-cinder-854070-1-iscsi devstack@keystone.service[81919]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 23 04:13:27.542732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 23 04:13:27.542732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: from cryptography.utils import int_from_bytes Aug 23 04:13:27.543548 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 23 04:13:27.543548 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: from cryptography.utils import int_from_bytes Aug 23 04:13:27.543656 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 23 04:13:27.543656 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: from cryptography.utils import int_from_bytes Aug 23 04:13:27.543656 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 23 04:13:27.543656 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: from cryptography.utils import int_from_bytes Aug 23 04:13:27.593101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 23 04:13:27.593306 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 23 04:13:27.593306 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 23 04:13:27.593449 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 23 04:13:27.593516 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] command line args: [] {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 23 04:13:27.593573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] command line args: [] {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 23 04:13:27.593710 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 23 04:13:27.593899 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 23 04:13:27.593973 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ================================================================================ {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 23 04:13:27.594042 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ================================================================================ {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 23 04:13:27.594105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] admin_token = **** {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.594171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] admin_token = **** {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.594308 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] config_dir = [] {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.594390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] config_dir = [] {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.594496 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.594578 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.594750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] config_source = [] {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.594890 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] config_source = [] {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.594971 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] debug = True {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.595037 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] debug = True {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.595095 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.595358 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.595457 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.595522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.595584 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.595646 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.595790 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.595874 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.596093 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.596225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.596298 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] list_limit = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.596361 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] list_limit = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.596484 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] log_config_append = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.596570 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] log_config_append = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.596635 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.596693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.596808 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] log_dir = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.596880 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] log_dir = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.596943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] log_file = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.597008 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] log_file = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.597205 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.597331 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.597402 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.597465 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.597679 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.597815 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.597949 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.598074 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.598074 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.598301 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.598425 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.598507 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.598566 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.598683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.598762 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.598834 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.598957 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.599033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.599097 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.599217 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.599344 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.599460 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.599609 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.599689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.599802 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.599946 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.600022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.600133 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.600327 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.600449 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.600843 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.601018 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.601153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] publish_errors = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.601277 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] publish_errors = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.601520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.601641 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.601761 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.601873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.601989 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.602098 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.602212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.602212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.602449 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.602560 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] standard_threads = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.602659 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.602772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.602887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] standard_threads = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.602999 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.603107 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.603220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.603220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] use_journal = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.603422 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.603584 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] use_json = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.603698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.603806 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] use_stderr = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.603967 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] use_journal = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.604094 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] use_syslog = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.604205 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] use_json = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.604205 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] use_stderr = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.604425 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.604540 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] use_syslog = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.604650 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.604759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 04:13:27.604873 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.604993 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.605106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.605215 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.605323 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.605430 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.605538 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.605654 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.605773 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.605882 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.605996 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.606107 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.606221 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.606338 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.606454 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.606563 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.606670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.606777 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.606890 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.607007 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.607117 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.607117 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.607325 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.607434 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.607547 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.607670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.607781 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.607889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.608014 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.608123 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.608231 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.608352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.608469 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.608579 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.608692 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.608802 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.608914 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.609104 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.609222 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.609333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.609442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.609552 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.609673 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.609783 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.609897 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.610006 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.610119 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.610228 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.610349 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.610459 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.610569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.610677 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.610795 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.610923 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.611046 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.611157 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.611264 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.611378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.611488 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.611596 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.611711 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.611821 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.611978 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.612091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.612209 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.612320 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.612437 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.612547 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.612655 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.612761 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.612761 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.612963 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.613079 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.613189 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.613297 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.613404 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.613512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.613619 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.613738 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.613848 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.613955 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.614063 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.614170 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.614284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.614471 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.614589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.614707 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.614816 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.614929 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.615037 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.615142 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.615235 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.615333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.615425 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.615522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.615615 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.615721 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.615824 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.615933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.616032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.616125 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.616232 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.616353 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.616464 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.616578 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.616687 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.616795 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.616902 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.617018 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.617119 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.617217 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.617361 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.617472 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.617472 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.617686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.617795 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.617903 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.618013 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.618130 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.618249 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.618359 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.618476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.618586 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.618687 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.618794 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.618911 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.619027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.619152 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.619263 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.619365 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.619483 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.619594 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.619703 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.619812 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.619963 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.620183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.620300 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.620401 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.620507 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.620631 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.620749 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.620862 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.620862 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.621074 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.621186 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.621302 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.621419 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.621529 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.621640 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.621749 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.621859 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.621976 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.622087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.622196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.622304 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.622412 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.622521 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.622638 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.622749 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.622858 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.623002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.623114 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.623224 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.623337 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.623447 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.623556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.623665 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.623774 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.623882 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.624027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.624147 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.624257 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.624368 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.624475 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.624584 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.624701 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.624817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.624927 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.625038 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.625146 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.625257 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.625445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.625558 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.625673 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.625782 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.625890 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.626008 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.626127 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.626246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.626356 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.626465 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.626648 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.626762 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.626872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.626994 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.627106 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.627216 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.627325 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.627435 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.627545 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.627662 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.627773 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.627882 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] auth.external = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.628035 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.628151 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.628261 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.628378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.628496 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.628605 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.628722 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.628841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] auth.password = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.628952 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.629091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] auth.token = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.629213 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] auth.external = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.629332 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.629443 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.629566 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.629566 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.629783 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.629894 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.630011 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.630123 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.630234 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] auth.password = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.630234 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] auth.token = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.630441 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.630441 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.630655 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.630774 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] credential.caching = True {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.630885 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.631003 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.631106 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.631208 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.631325 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.631437 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.631546 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.631656 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.631765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.631881 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.632022 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.632136 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.632246 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] credential.caching = True {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.632356 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.632466 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.632575 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.632750 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.632865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.633044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.633044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.633353 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.633465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.633588 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.633707 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.633819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.633929 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.634039 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.634155 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.634275 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.634387 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.634501 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.634611 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.634721 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] federation.caching = True {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.634831 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.634949 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.635068 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.635179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.635179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.635395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.635507 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.635626 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.635738 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.635854 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.635987 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.636100 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] federation.caching = True {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.636224 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.636338 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.636449 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.636560 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.636671 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.636781 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.636900 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.637012 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.637123 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.637233 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.637344 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] identity.caching = True {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.637455 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.637573 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.637686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.637797 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.637913 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.638025 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.638142 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.638262 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.638375 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.638490 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.638602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.638721 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.638840 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] identity.caching = True {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.639034 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.639149 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.639259 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.639369 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.639479 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.639590 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.639710 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.639822 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.639978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.640092 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.640203 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.640323 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.640445 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.640557 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.640667 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.640778 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.640888 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.640997 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.641114 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.641235 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.641346 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.641456 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.641567 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.641683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.641804 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.641916 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.642026 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.642135 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.642244 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.642353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.642472 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.642472 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.642680 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.642790 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.642901 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.643011 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.643130 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.643242 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.643352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.643461 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.643570 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.643680 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.643798 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.643923 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.644036 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.644147 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.644257 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.644366 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.644489 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.644602 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.644712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.644821 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.644932 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.645042 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.645160 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.645272 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.645385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.645495 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.645604 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.645726 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.645838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.645948 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.646057 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.646166 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.646275 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.646392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.646504 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.646614 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.646724 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.646833 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.646943 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.647065 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.647177 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.647289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.647399 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.647508 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.647617 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.647735 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.647847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.647998 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.648112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.648260 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.648375 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.648498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.648611 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.648721 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.648830 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.648941 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.user = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.649050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.649169 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.649282 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.649392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.649502 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.649612 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.649722 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.649841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.649952 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.650062 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.650173 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.650283 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.650393 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.650512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.650624 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.user = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.650733 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.650843 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.650953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.650953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.651168 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.651287 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.651398 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.651509 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.651619 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.651734 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.651853 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.651981 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.652092 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.652203 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.652314 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.652424 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.652546 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.652659 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.652769 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.652880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.652990 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.653100 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.653218 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.653331 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.653441 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.653551 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.653660 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.653771 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.653889 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.654001 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.654111 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.654222 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.654331 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.654443 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.654564 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.654677 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.654787 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.654898 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.655007 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.655117 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.655235 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.655347 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.655458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.655568 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.655678 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.655793 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.655965 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.656082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.656192 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.656304 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.656413 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.656524 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.656645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.656758 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.656868 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.656978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.657087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.657198 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] resource.caching = True {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.657310 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.657422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.657533 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.657642 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.657752 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.657862 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.657979 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.658090 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.658201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.658311 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.658419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.658529 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.658647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.658759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.658868 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.658978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.659087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] resource.caching = True {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.659197 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.659315 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.659428 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] role.caching = True {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.659538 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.659648 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] role.driver = sql {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.659757 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.659873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.660017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.660128 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.660238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.660349 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.660450 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.660636 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.660752 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.660863 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.660973 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.661084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.661195 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.661313 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.661425 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.661535 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] role.caching = True {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.661645 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.661755 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] role.driver = sql {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.661866 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.661983 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.662095 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.662205 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.662315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.662424 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.662534 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.662652 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.662766 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.662876 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.663020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.663134 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.663279 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.663402 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.663514 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.663623 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.663758 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.664014 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.664182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.664313 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.664427 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.664666 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.664782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.664895 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.665008 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.665149 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.665273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.665386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.665541 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.665656 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.665775 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.665889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.666005 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.666116 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.666262 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.666377 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.666497 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.666608 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.666753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.666867 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.666979 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.667089 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.667209 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.667321 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.667432 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.667542 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.667653 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.667763 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.667881 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] token.caching = True {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.668015 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.668127 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.668238 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.668348 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.668461 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.668584 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.668698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.668809 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.668920 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.669102 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.669219 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.669340 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.669453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.669565 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.669675 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.669786 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.669896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.670015 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.670128 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] token.caching = True {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.670239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.670349 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.670460 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.670570 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.670689 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.670801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.670911 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.671021 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.671133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.671243 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.671361 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.671473 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.671583 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.671678 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.671807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.671976 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.672105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] database.connection = **** {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.672230 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.672351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.672464 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.672581 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.672692 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.672803 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.672921 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.673033 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.673144 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.673255 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.673365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.673475 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.673593 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.673705 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.673822 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.673934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.674044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.674155 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.674273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] database.connection = **** {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.674394 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.674517 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.674630 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.674741 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.674851 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.674967 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.675079 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.675198 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.675311 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.675422 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.675533 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.675643 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.675754 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.675873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.676008 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.676120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.676229 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.676339 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 23 04:13:27.676450 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.676596 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.676843 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.677082 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.677333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.677570 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.677818 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.678052 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.678294 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.678532 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 04:13:27.678706 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 23 04:13:27.705814 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:27.706042 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:27.706310 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:27.707426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:27.707665 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:27.707894 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:27.715271 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:27.716857 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:27.743406 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:27.743582 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:27.746734 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:27.746943 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:27.796274 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:27.796954 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:27.844577 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:27.844785 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:27.845405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:27.845638 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.007843 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.008050 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.008133 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.008195 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.008280 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.008344 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.008404 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.008463 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.008621 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.008707 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.008834 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.008921 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.032060 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.032198 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.032275 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.032424 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.032506 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.032686 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.032811 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.032888 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.032969 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.033109 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.033245 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.033384 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.033464 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.033541 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.044719 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.044848 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.057659 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.057922 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.058063 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.058216 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.058304 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.058367 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.058432 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.058528 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.058623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.058731 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.058808 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.058970 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.059097 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.059248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 04:13:28.075611 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.075820 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 23 04:13:28.076176 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.076300 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.076422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 23 04:13:28.076525 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 23 04:13:28.076650 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.076844 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.076979 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.077115 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 23 04:13:28.077325 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.077449 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.077586 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.077665 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.078099 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.078343 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.086947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.087224 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.087491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.087774 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.087899 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.088095 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.088359 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.088647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.095106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.096043 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.098020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.098307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.098586 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.098864 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.098988 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.099136 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.099228 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.099337 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.099534 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.099658 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.099793 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.099886 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.100065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.100298 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.100431 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.100615 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.100808 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.100941 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.101068 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.101232 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.101847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.102044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.123433 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.124859 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.124986 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.125121 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.126327 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.126617 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.129806 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.130088 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.130366 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.131348 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.131643 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.131956 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.136839 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.137128 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.138349 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.138641 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.141226 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.141509 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.141790 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.142072 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.142345 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.142614 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.142738 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.142899 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.143032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.143122 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.143345 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.143631 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.143941 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.144224 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.144499 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.144756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.160603 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.160879 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.161908 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.162194 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.164836 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.166145 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.168065 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.168361 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.169417 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.169705 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.172567 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.173972 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.175685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.177152 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.178936 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.179236 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.179560 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.179825 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.180122 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.180406 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.180522 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.180813 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.181104 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.181393 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.181693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.181987 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.196564 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.196850 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.197112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.197386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.197940 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.198236 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.198510 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.198791 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.204264 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.205706 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.205852 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.205992 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.207240 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.207524 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.208674 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.208943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.209255 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.209544 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.209819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.210188 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.210476 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.210888 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.211187 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.211481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.221852 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.222135 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.222414 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.222690 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.222971 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.223243 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.223387 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.223530 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.223661 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.223943 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.224284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.224571 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.224861 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.225151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.238954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.240848 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.242024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.243956 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.244958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.246388 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.246890 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.247851 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.248157 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.248374 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.248475 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.249837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.250130 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.250425 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.256208 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.256987 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.258010 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.258484 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.261461 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.261788 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.262071 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.262197 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.262333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.262451 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.265301 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.265597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.270335 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.270635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.270924 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.273627 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.273916 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.274167 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.277996 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.278284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.278569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.278851 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.279141 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.279433 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.279715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.279994 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.280291 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.280576 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.280854 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.281069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 04:13:28.281154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.281356 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.281453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.281614 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.281693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.281885 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.282157 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.282446 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.282720 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.282986 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.283268 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.283537 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.283815 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.284103 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.284374 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.284632 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 04:13:28.315908 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55888d4199c0 pid: 81934 (default app) Aug 23 04:13:28.318972 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55888d4199c0 pid: 81933 (default app) Aug 23 04:13:28.319386 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826d32c-4eb3-4422-8fcd-0ad2b9c16212 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:28.319673 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826d32c-4eb3-4422-8fcd-0ad2b9c16212 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:28.319956 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826d32c-4eb3-4422-8fcd-0ad2b9c16212 None None] PATH_INFO: `/v3` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:28.324209 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1/1] 10.222.0.18 () {50 vars in 791 bytes} [Tue Aug 23 04:13:28 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:34.933288 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93767b30-adb8-4e15-b5a4-c9c2c3e748e4 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:34.933822 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93767b30-adb8-4e15-b5a4-c9c2c3e748e4 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:34.933822 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93767b30-adb8-4e15-b5a4-c9c2c3e748e4 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:34.938124 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1/2] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:13:34 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:13:34.945886 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5bdba-1368-49ec-873f-7222fec3725b None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:34.946603 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5bdba-1368-49ec-873f-7222fec3725b None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:34.946685 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5bdba-1368-49ec-873f-7222fec3725b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:35.023838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: INFO dbcounter [None req-42d5bdba-1368-49ec-873f-7222fec3725b None None] Registered counter for database keystone Aug 23 04:13:35.045804 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG oslo_db.sqlalchemy.engines [None req-42d5bdba-1368-49ec-873f-7222fec3725b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81934) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 23 04:13:35.053810 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writer thread running {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 23 04:13:35.252185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG passlib.handlers.bcrypt [None req-42d5bdba-1368-49ec-873f-7222fec3725b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81934) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 23 04:13:35.252561 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG passlib.handlers.bcrypt [None req-42d5bdba-1368-49ec-873f-7222fec3725b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81934) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 23 04:13:35.307753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-42d5bdba-1368-49ec-873f-7222fec3725b None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:13:35.309402 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-42d5bdba-1368-49ec-873f-7222fec3725b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:13:35.446561 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 2/3] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:13:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 502 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 23 04:13:35.929994 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93767b30-adb8-4e15-b5a4-c9c2c3e748e4 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:36.006076 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: INFO dbcounter [None req-93767b30-adb8-4e15-b5a4-c9c2c3e748e4 None None] Registered counter for database keystone Aug 23 04:13:36.030974 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG oslo_db.sqlalchemy.engines [None req-93767b30-adb8-4e15-b5a4-c9c2c3e748e4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81933) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 23 04:13:36.042095 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writer thread running {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 23 04:13:36.160713 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93767b30-adb8-4e15-b5a4-c9c2c3e748e4 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:36.166290 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c64044-bf21-4aa9-ac2a-0d76ea3df5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:36.166928 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c64044-bf21-4aa9-ac2a-0d76ea3df5e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:36.167207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c64044-bf21-4aa9-ac2a-0d76ea3df5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:36.167474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c64044-bf21-4aa9-ac2a-0d76ea3df5e6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:36.176568 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c64044-bf21-4aa9-ac2a-0d76ea3df5e6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:36.229166 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: WARNING keystone.common.rbac_enforcer.enforcer [None req-73c64044-bf21-4aa9-ac2a-0d76ea3df5e6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 23 04:13:36.229435 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c64044-bf21-4aa9-ac2a-0d76ea3df5e6 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:36.240735 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 2/4] 10.222.0.18 () {58 vars in 1324 bytes} [Tue Aug 23 04:13:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 04:13:36.247970 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5bdba-1368-49ec-873f-7222fec3725b None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:36.272625 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5bdba-1368-49ec-873f-7222fec3725b None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:36.279685 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b48f80-4943-49be-8374-9c1aa807c348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:36.280850 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b48f80-4943-49be-8374-9c1aa807c348 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:36.281128 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b48f80-4943-49be-8374-9c1aa807c348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:36.281390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b48f80-4943-49be-8374-9c1aa807c348 admin admin] PATH_INFO: `/v3/projects/ba69f2ba4f0d496e9c7607a800699841` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:36.283078 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b48f80-4943-49be-8374-9c1aa807c348 admin admin] RBAC: Authorizing `identity:get_project(project_id=ba69f2ba4f0d496e9c7607a800699841)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:36.336149 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: WARNING keystone.common.rbac_enforcer.enforcer [None req-83b48f80-4943-49be-8374-9c1aa807c348 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 23 04:13:36.336490 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b48f80-4943-49be-8374-9c1aa807c348 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:36.341415 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 3/5] 10.222.0.18 () {56 vars in 1207 bytes} [Tue Aug 23 04:13:36 2022] GET /identity/v3/projects/ba69f2ba4f0d496e9c7607a800699841 => generated 344 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:38.265425 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20c975-ca36-488d-8865-64052b09d8b5 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:38.265972 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20c975-ca36-488d-8865-64052b09d8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:38.265972 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20c975-ca36-488d-8865-64052b09d8b5 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:38.270059 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 3/6] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:13:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:13:38.277055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae6bae-974d-4298-b713-748d9157aef8 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:38.277763 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae6bae-974d-4298-b713-748d9157aef8 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:38.277902 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae6bae-974d-4298-b713-748d9157aef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:38.305429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-8eae6bae-974d-4298-b713-748d9157aef8 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:13:38.306975 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-8eae6bae-974d-4298-b713-748d9157aef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:13:38.316288 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 4/7] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:13:38 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 23 04:13:38.591402 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20c975-ca36-488d-8865-64052b09d8b5 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:38.606189 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20c975-ca36-488d-8865-64052b09d8b5 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:38.611580 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f7d5c-5381-4b77-b82b-8b8f04390385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:38.612204 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f7d5c-5381-4b77-b82b-8b8f04390385 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:38.612478 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f7d5c-5381-4b77-b82b-8b8f04390385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:38.612750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f7d5c-5381-4b77-b82b-8b8f04390385 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:38.620906 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f7d5c-5381-4b77-b82b-8b8f04390385 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:38.622683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f7d5c-5381-4b77-b82b-8b8f04390385 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:38.631427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 4/8] 10.222.0.18 () {58 vars in 1324 bytes} [Tue Aug 23 04:13:38 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 04:13:38.637476 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae6bae-974d-4298-b713-748d9157aef8 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:38.651589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae6bae-974d-4298-b713-748d9157aef8 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:38.657567 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb59bb7-9d5a-4edf-96b5-0cc4935f66c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:38.658217 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb59bb7-9d5a-4edf-96b5-0cc4935f66c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:38.658496 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb59bb7-9d5a-4edf-96b5-0cc4935f66c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:38.658766 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb59bb7-9d5a-4edf-96b5-0cc4935f66c0 admin admin] PATH_INFO: `/v3/users/de65eb39f13f4a92a6a526fb5ab1f5c5` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:38.660333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb59bb7-9d5a-4edf-96b5-0cc4935f66c0 admin admin] RBAC: Authorizing `identity:get_user(user_id=de65eb39f13f4a92a6a526fb5ab1f5c5)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:38.662190 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb59bb7-9d5a-4edf-96b5-0cc4935f66c0 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:38.663730 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 5/9] 10.222.0.18 () {56 vars in 1198 bytes} [Tue Aug 23 04:13:38 2022] GET /identity/v3/users/de65eb39f13f4a92a6a526fb5ab1f5c5 => generated 247 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:40.624654 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63b132-4189-419d-a81f-d066e1dcd259 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:40.625226 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63b132-4189-419d-a81f-d066e1dcd259 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:40.625226 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63b132-4189-419d-a81f-d066e1dcd259 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:40.626538 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 5/10] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:13:40.631786 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825f8e01-089d-456c-ace7-9bd66cac217c None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:40.632282 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825f8e01-089d-456c-ace7-9bd66cac217c None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:40.632384 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825f8e01-089d-456c-ace7-9bd66cac217c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:40.647579 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d75ab0-7ea1-4802-81a9-cb584e927a69 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:40.647863 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d75ab0-7ea1-4802-81a9-cb584e927a69 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:40.648184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d75ab0-7ea1-4802-81a9-cb584e927a69 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:40.649603 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 6/11] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:13:40.651699 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-825f8e01-089d-456c-ace7-9bd66cac217c None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:13:40.652942 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-825f8e01-089d-456c-ace7-9bd66cac217c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:13:40.655282 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeedd4a-8b6a-4a45-ae5d-6bd4084a1198 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:40.655565 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeedd4a-8b6a-4a45-ae5d-6bd4084a1198 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:40.655843 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeedd4a-8b6a-4a45-ae5d-6bd4084a1198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:40.698114 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG passlib.handlers.bcrypt [None req-ebeedd4a-8b6a-4a45-ae5d-6bd4084a1198 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81933) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 23 04:13:40.698531 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG passlib.handlers.bcrypt [None req-ebeedd4a-8b6a-4a45-ae5d-6bd4084a1198 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81933) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 23 04:13:40.740876 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 6/12] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:13:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:13:40.744103 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-ebeedd4a-8b6a-4a45-ae5d-6bd4084a1198 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:13:40.745783 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-ebeedd4a-8b6a-4a45-ae5d-6bd4084a1198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:13:40.846711 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 7/13] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:13:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:13:41.007564 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825f8e01-089d-456c-ace7-9bd66cac217c None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:41.032464 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:41.113089 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeedd4a-8b6a-4a45-ae5d-6bd4084a1198 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:41.115604 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825f8e01-089d-456c-ace7-9bd66cac217c None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:41.153997 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50244272-cbec-4b36-8fe0-ea2ae45c7f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:41.154643 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50244272-cbec-4b36-8fe0-ea2ae45c7f69 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:41.154923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50244272-cbec-4b36-8fe0-ea2ae45c7f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:41.155200 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50244272-cbec-4b36-8fe0-ea2ae45c7f69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:41.166349 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50244272-cbec-4b36-8fe0-ea2ae45c7f69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:13:41.175081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-50244272-cbec-4b36-8fe0-ea2ae45c7f69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:13:41.175081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:13:41.175081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:13:41.175081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:13:41.175081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:13:41.175081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:13:41.175081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:13:41.175081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:13:41.175081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:13:41.175081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:13:41.175081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:13:41.175081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:13:41.175081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:13:41.175081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:13:41.175081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:13:41.175081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:13:41.176034 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:13:41.176034 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:13:41.176034 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:13:41.176034 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:13:41.176034 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:13:41.176034 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:13:41.176034 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:13:41.176034 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:13:41.176034 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:13:41.176034 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:13:41.176034 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:13:41.176034 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:13:41.176034 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:13:41.176034 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:13:41.176034 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:13:41.176034 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:13:41.176820 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:13:41.176820 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:13:41.176820 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:13:41.176820 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:13:41.176820 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:13:41.176820 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:13:41.176820 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:13:41.176820 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:13:41.176820 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:13:41.176820 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:13:41.176820 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:13:41.176820 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:13:41.176820 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 7/14] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:13:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:13:41.182275 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50244272-cbec-4b36-8fe0-ea2ae45c7f69 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:41.222842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeedd4a-8b6a-4a45-ae5d-6bd4084a1198 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:41.225393 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:41.261568 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d390c-3859-42ea-b16a-52ce0665cf7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:41.262357 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d390c-3859-42ea-b16a-52ce0665cf7a None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:41.262633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d390c-3859-42ea-b16a-52ce0665cf7a None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:41.262899 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d390c-3859-42ea-b16a-52ce0665cf7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:41.268826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:41.287150 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50244272-cbec-4b36-8fe0-ea2ae45c7f69 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:41.321455 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d4f8f8-8124-4f5a-8c00-a642f254fa47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:41.322148 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4f8f8-8124-4f5a-8c00-a642f254fa47 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:41.322436 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4f8f8-8124-4f5a-8c00-a642f254fa47 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:41.322708 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4f8f8-8124-4f5a-8c00-a642f254fa47 None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:41.323391 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d4f8f8-8124-4f5a-8c00-a642f254fa47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:13:41.323696 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d4f8f8-8124-4f5a-8c00-a642f254fa47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:41.325174 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d4f8f8-8124-4f5a-8c00-a642f254fa47 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:41.331250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263d390c-3859-42ea-b16a-52ce0665cf7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:41.332816 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263d390c-3859-42ea-b16a-52ce0665cf7a None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:41.336931 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 8/15] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:13:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:13:41.343393 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d4f8f8-8124-4f5a-8c00-a642f254fa47 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:41.433052 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 8/16] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:13:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:13:41.438607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d390c-3859-42ea-b16a-52ce0665cf7a None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:41.441287 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:41.448759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d4f8f8-8124-4f5a-8c00-a642f254fa47 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:41.490683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c1def-eef5-4e22-a359-8752f5e86add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:41.491319 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c1def-eef5-4e22-a359-8752f5e86add None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:41.491595 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c1def-eef5-4e22-a359-8752f5e86add None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:41.491861 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c1def-eef5-4e22-a359-8752f5e86add None admin] PATH_INFO: `/v3/users/de65eb39f13f4a92a6a526fb5ab1f5c5` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:41.493048 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2c1def-eef5-4e22-a359-8752f5e86add None admin] RBAC: Authorizing `identity:get_user(user_id=de65eb39f13f4a92a6a526fb5ab1f5c5)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:41.494555 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2c1def-eef5-4e22-a359-8752f5e86add None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:41.496140 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 9/17] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:13:41 2022] GET /identity/v3/users/de65eb39f13f4a92a6a526fb5ab1f5c5 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:13:41.501740 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c1def-eef5-4e22-a359-8752f5e86add None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:41.516650 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:41.539355 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d390c-3859-42ea-b16a-52ce0665cf7a None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:41.572480 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05d410-6513-420d-b1ec-73fec2d7b5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:41.573336 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05d410-6513-420d-b1ec-73fec2d7b5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:41.573616 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05d410-6513-420d-b1ec-73fec2d7b5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:41.573887 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05d410-6513-420d-b1ec-73fec2d7b5e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:41.592742 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd05d410-6513-420d-b1ec-73fec2d7b5e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:41.594230 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd05d410-6513-420d-b1ec-73fec2d7b5e9 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:41.599804 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c1def-eef5-4e22-a359-8752f5e86add None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:41.612167 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-bd05d410-6513-420d-b1ec-73fec2d7b5e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:13:41.612167 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:13:41.612167 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:13:41.612167 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:13:41.612167 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:13:41.612167 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:13:41.612167 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:13:41.612167 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:13:41.612167 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:13:41.612167 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:13:41.612167 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:13:41.612167 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:13:41.612167 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:13:41.612167 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:13:41.612167 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:13:41.612167 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:13:41.613207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:13:41.613207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:13:41.613207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:13:41.613207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:13:41.613207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:13:41.613207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:13:41.613207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:13:41.613207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:13:41.613207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:13:41.613207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:13:41.613207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:13:41.613207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:13:41.613207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:13:41.613207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:13:41.613207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:13:41.613207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:13:41.613207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:13:41.614141 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:13:41.614141 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:13:41.614141 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:13:41.614141 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 9/18] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:13:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:13:41.619600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05d410-6513-420d-b1ec-73fec2d7b5e9 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:41.636213 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c114a3-d0c2-42d1-8d40-0c253d7e6e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:41.636989 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c114a3-d0c2-42d1-8d40-0c253d7e6e26 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:41.637131 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c114a3-d0c2-42d1-8d40-0c253d7e6e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:41.637362 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c114a3-d0c2-42d1-8d40-0c253d7e6e26 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:41.638529 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c114a3-d0c2-42d1-8d40-0c253d7e6e26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:41.640013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c114a3-d0c2-42d1-8d40-0c253d7e6e26 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:41.641538 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 10/19] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:13:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:13:41.647469 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c114a3-d0c2-42d1-8d40-0c253d7e6e26 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:41.678056 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:41.722560 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05d410-6513-420d-b1ec-73fec2d7b5e9 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:41.728761 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:41.748203 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c114a3-d0c2-42d1-8d40-0c253d7e6e26 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:41.755932 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c1adf-d58b-4b0e-9735-49d82676bfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:41.756556 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c1adf-d58b-4b0e-9735-49d82676bfcd None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:41.756843 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c1adf-d58b-4b0e-9735-49d82676bfcd None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:41.757115 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c1adf-d58b-4b0e-9735-49d82676bfcd None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:41.757771 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c1adf-d58b-4b0e-9735-49d82676bfcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:13:41.758075 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c1adf-d58b-4b0e-9735-49d82676bfcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:41.759516 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c1adf-d58b-4b0e-9735-49d82676bfcd None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:41.778020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 10/20] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:41.781886 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0453e89-13c2-40d0-95f5-065bdcacbbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:41.782517 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0453e89-13c2-40d0-95f5-065bdcacbbe3 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:41.782802 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0453e89-13c2-40d0-95f5-065bdcacbbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:41.783081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0453e89-13c2-40d0-95f5-065bdcacbbe3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:41.783830 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0453e89-13c2-40d0-95f5-065bdcacbbe3 None admin] RBAC: Adding query filter params (role.id=52b31061b72c45ccb76dd26bf31f8d3f, scope.domain.id=default, user.id=de65eb39f13f4a92a6a526fb5ab1f5c5) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:13:41.784175 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0453e89-13c2-40d0-95f5-065bdcacbbe3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:41.785681 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0453e89-13c2-40d0-95f5-065bdcacbbe3 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:41.793983 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 11/21] 10.222.0.18 () {56 vars in 1322 bytes} [Tue Aug 23 04:13:41 2022] GET /identity/v3/role_assignments?user.id=de65eb39f13f4a92a6a526fb5ab1f5c5&scope.domain.id=default&role.id=52b31061b72c45ccb76dd26bf31f8d3f => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:43.896016 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5e9cc-b53a-4b2d-9d15-a03db1005980 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:43.896510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5e9cc-b53a-4b2d-9d15-a03db1005980 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:43.896510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5e9cc-b53a-4b2d-9d15-a03db1005980 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:43.897816 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 11/22] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:13:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:13:43.903778 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0aeef-ede0-4c2a-8b98-d34f0318ac12 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:43.904230 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0aeef-ede0-4c2a-8b98-d34f0318ac12 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:43.904418 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0aeef-ede0-4c2a-8b98-d34f0318ac12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:43.926230 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-4ba0aeef-ede0-4c2a-8b98-d34f0318ac12 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:13:43.927439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-4ba0aeef-ede0-4c2a-8b98-d34f0318ac12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:13:44.019678 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 12/23] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:13:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:13:44.174763 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f375a0f-b0f0-4f0e-a949-fc08c6ff0bc7 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:44.175111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f375a0f-b0f0-4f0e-a949-fc08c6ff0bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:44.175324 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f375a0f-b0f0-4f0e-a949-fc08c6ff0bc7 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:44.176873 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 12/24] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:13:44.182723 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24898f7d-c991-49c4-9a95-35156c999571 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:44.183014 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24898f7d-c991-49c4-9a95-35156c999571 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:44.183292 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24898f7d-c991-49c4-9a95-35156c999571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:44.208918 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-24898f7d-c991-49c4-9a95-35156c999571 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:13:44.210109 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-24898f7d-c991-49c4-9a95-35156c999571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:13:44.297100 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 13/25] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:13:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:13:44.432712 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e06abb-f088-4f48-8043-135718b484ea None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:44.433048 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e06abb-f088-4f48-8043-135718b484ea None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:44.433211 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e06abb-f088-4f48-8043-135718b484ea None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:44.434692 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 13/26] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:13:44.447955 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d77b8-7d2e-4dfc-88d3-9785321afe5a None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:44.448249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d77b8-7d2e-4dfc-88d3-9785321afe5a None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:44.448529 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d77b8-7d2e-4dfc-88d3-9785321afe5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:44.463747 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:44.481457 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-c56d77b8-7d2e-4dfc-88d3-9785321afe5a None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:13:44.482659 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-c56d77b8-7d2e-4dfc-88d3-9785321afe5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:13:44.518975 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e06abb-f088-4f48-8043-135718b484ea None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:44.638439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 14/27] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:13:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:13:44.680918 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c4d5af-e43e-4586-96fe-e1cfd755f6e7 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:44.681262 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c4d5af-e43e-4586-96fe-e1cfd755f6e7 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:44.681488 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c4d5af-e43e-4586-96fe-e1cfd755f6e7 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:44.683050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 15/28] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:13:44.692566 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ce632-1cd0-49ab-a0a2-ab92aa17b8cb None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:44.692854 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ce632-1cd0-49ab-a0a2-ab92aa17b8cb None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:44.693136 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ce632-1cd0-49ab-a0a2-ab92aa17b8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:44.710529 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e06abb-f088-4f48-8043-135718b484ea None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:44.735879 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-380ce632-1cd0-49ab-a0a2-ab92aa17b8cb None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:13:44.737132 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-380ce632-1cd0-49ab-a0a2-ab92aa17b8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:13:44.752326 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:44.783419 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ff9b2-a670-4146-b7f2-26c2671997a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:44.784117 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ff9b2-a670-4146-b7f2-26c2671997a5 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:44.784400 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ff9b2-a670-4146-b7f2-26c2671997a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:44.784735 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ff9b2-a670-4146-b7f2-26c2671997a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:44.809522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ff9b2-a670-4146-b7f2-26c2671997a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:13:44.816112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-6a6ff9b2-a670-4146-b7f2-26c2671997a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:13:44.816112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:13:44.816112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:13:44.816112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:13:44.816112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:13:44.816112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:13:44.816112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:13:44.816112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:13:44.816112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:13:44.816112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:13:44.816112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:13:44.816112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:13:44.816112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:13:44.816112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:13:44.816112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:13:44.816112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:13:44.817091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:13:44.817091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:13:44.817091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:13:44.817091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:13:44.817091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:13:44.817091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:13:44.817091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:13:44.817091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:13:44.817091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:13:44.817091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:13:44.817091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:13:44.817091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:13:44.817091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:13:44.817091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:13:44.817091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:13:44.817091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:13:44.818010 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:13:44.818010 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:13:44.818010 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:13:44.818010 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:13:44.818010 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:13:44.818010 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:13:44.818010 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:13:44.818010 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:13:44.818010 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:13:44.818010 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:13:44.818010 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:13:44.818010 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:13:44.820476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 14/29] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:13:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:13:44.822468 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ff9b2-a670-4146-b7f2-26c2671997a5 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:44.869279 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 16/30] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:13:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 04:13:44.872019 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380ce632-1cd0-49ab-a0a2-ab92aa17b8cb None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:45.000492 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:45.016643 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ff9b2-a670-4146-b7f2-26c2671997a5 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:45.051718 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342322f8-cb76-48d4-82d6-24e345800db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:45.052616 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342322f8-cb76-48d4-82d6-24e345800db7 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:45.053070 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342322f8-cb76-48d4-82d6-24e345800db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:45.053493 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342322f8-cb76-48d4-82d6-24e345800db7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:45.053875 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380ce632-1cd0-49ab-a0a2-ab92aa17b8cb None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:45.065419 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342322f8-cb76-48d4-82d6-24e345800db7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:45.067511 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342322f8-cb76-48d4-82d6-24e345800db7 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:45.083110 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-342322f8-cb76-48d4-82d6-24e345800db7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:13:45.083110 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:13:45.083110 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:13:45.083110 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:13:45.083110 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:13:45.083110 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:13:45.083110 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:13:45.083110 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:13:45.083110 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:13:45.083110 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:13:45.083110 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:13:45.083110 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:13:45.083110 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:13:45.083110 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:13:45.083110 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:13:45.083110 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:13:45.084158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:13:45.084158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:13:45.084158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:13:45.084158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:13:45.084158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:13:45.084158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:13:45.084158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:13:45.084158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:13:45.084158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:13:45.084158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:13:45.084158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:13:45.084158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:13:45.084158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:13:45.084158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:13:45.084158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:13:45.084158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:13:45.084158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:13:45.085212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:13:45.085212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:13:45.085212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:13:45.086772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 15/31] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:13:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:13:45.088150 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342322f8-cb76-48d4-82d6-24e345800db7 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:45.120317 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:45.178104 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2fcf2-18b3-4b84-8fd6-f15fcb85b991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:45.178749 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2fcf2-18b3-4b84-8fd6-f15fcb85b991 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:45.179034 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2fcf2-18b3-4b84-8fd6-f15fcb85b991 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:45.179306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2fcf2-18b3-4b84-8fd6-f15fcb85b991 None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:45.180009 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d2fcf2-18b3-4b84-8fd6-f15fcb85b991 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:13:45.180561 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d2fcf2-18b3-4b84-8fd6-f15fcb85b991 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:45.182227 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d2fcf2-18b3-4b84-8fd6-f15fcb85b991 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:45.196254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342322f8-cb76-48d4-82d6-24e345800db7 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:45.206094 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 17/32] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:13:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:45.206952 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2fcf2-18b3-4b84-8fd6-f15fcb85b991 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:45.233903 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ef74b-cd69-49ae-9843-0b7ebd1a2c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:45.234865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ef74b-cd69-49ae-9843-0b7ebd1a2c6d None admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:45.235360 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ef74b-cd69-49ae-9843-0b7ebd1a2c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:45.235825 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ef74b-cd69-49ae-9843-0b7ebd1a2c6d None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:45.236753 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362ef74b-cd69-49ae-9843-0b7ebd1a2c6d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:45.238347 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362ef74b-cd69-49ae-9843-0b7ebd1a2c6d None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:45.323880 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2fcf2-18b3-4b84-8fd6-f15fcb85b991 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:45.351867 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:45.361527 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9f575-daff-4c44-9982-7eda7362aeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:45.362143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9f575-daff-4c44-9982-7eda7362aeea None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:45.362419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9f575-daff-4c44-9982-7eda7362aeea None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:45.362689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9f575-daff-4c44-9982-7eda7362aeea None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:45.363331 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e9f575-daff-4c44-9982-7eda7362aeea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:13:45.363687 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e9f575-daff-4c44-9982-7eda7362aeea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:45.365124 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e9f575-daff-4c44-9982-7eda7362aeea None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:45.383589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 18/33] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:45.384455 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9f575-daff-4c44-9982-7eda7362aeea None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:45.389882 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 16/34] 10.222.0.18 () {60 vars in 1131 bytes} [Tue Aug 23 04:13:45 2022] POST /identity/v3/roles => generated 225 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:13:45.390807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ef74b-cd69-49ae-9843-0b7ebd1a2c6d None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:45.501683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:45.507366 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9f575-daff-4c44-9982-7eda7362aeea None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:45.516192 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ef74b-cd69-49ae-9843-0b7ebd1a2c6d None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:45.549798 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eda20e-ad05-46f1-907e-01150d0c6563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:45.551229 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eda20e-ad05-46f1-907e-01150d0c6563 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:45.551706 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eda20e-ad05-46f1-907e-01150d0c6563 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:45.553290 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eda20e-ad05-46f1-907e-01150d0c6563 None admin] PATH_INFO: `/v3/users/de65eb39f13f4a92a6a526fb5ab1f5c5` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:45.554835 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eda20e-ad05-46f1-907e-01150d0c6563 None admin] RBAC: Authorizing `identity:get_user(user_id=de65eb39f13f4a92a6a526fb5ab1f5c5)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:45.556655 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eda20e-ad05-46f1-907e-01150d0c6563 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:45.557917 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d620b72f-a279-4252-b322-586440928706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:45.558555 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620b72f-a279-4252-b322-586440928706 None admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:45.558841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620b72f-a279-4252-b322-586440928706 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:45.559118 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620b72f-a279-4252-b322-586440928706 None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:45.559335 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 19/35] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:13:45 2022] GET /identity/v3/users/de65eb39f13f4a92a6a526fb5ab1f5c5 => generated 247 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:45.560589 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d620b72f-a279-4252-b322-586440928706 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:45.560896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08123ef-f98a-4f07-a172-bc9de83f2745 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:45.561359 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08123ef-f98a-4f07-a172-bc9de83f2745 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:45.561810 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08123ef-f98a-4f07-a172-bc9de83f2745 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:45.562248 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d620b72f-a279-4252-b322-586440928706 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:45.563452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 20/36] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:13:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:13:45.564377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08123ef-f98a-4f07-a172-bc9de83f2745 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:45.581509 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 17/37] 10.222.0.18 () {60 vars in 1131 bytes} [Tue Aug 23 04:13:45 2022] POST /identity/v3/roles => generated 219 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:13:45.587039 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72938c1-24ac-4221-a9b8-819d1b416283 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:45.587039 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72938c1-24ac-4221-a9b8-819d1b416283 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:45.587039 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72938c1-24ac-4221-a9b8-819d1b416283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:45.610515 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-b72938c1-24ac-4221-a9b8-819d1b416283 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:13:45.612557 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-b72938c1-24ac-4221-a9b8-819d1b416283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:13:45.638514 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:45.693387 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08123ef-f98a-4f07-a172-bc9de83f2745 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:45.700825 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 18/38] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:13:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:13:45.701843 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72938c1-24ac-4221-a9b8-819d1b416283 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:45.734290 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aaa3f2-38e4-4b31-a91a-0dd160e23482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:45.734926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aaa3f2-38e4-4b31-a91a-0dd160e23482 None admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:45.735207 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aaa3f2-38e4-4b31-a91a-0dd160e23482 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:45.735482 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aaa3f2-38e4-4b31-a91a-0dd160e23482 None admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:45.736266 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aaa3f2-38e4-4b31-a91a-0dd160e23482 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:45.737883 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aaa3f2-38e4-4b31-a91a-0dd160e23482 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:45.833508 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:45.861226 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72938c1-24ac-4221-a9b8-819d1b416283 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:45.903170 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c9d20-39f3-4d28-9dab-9eab0cb43be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:45.903845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c9d20-39f3-4d28-9dab-9eab0cb43be4 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:45.904157 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c9d20-39f3-4d28-9dab-9eab0cb43be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:45.904429 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c9d20-39f3-4d28-9dab-9eab0cb43be4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:45.905868 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c9d20-39f3-4d28-9dab-9eab0cb43be4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:45.907348 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c9d20-39f3-4d28-9dab-9eab0cb43be4 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:45.909245 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 19/39] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:13:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:45.913296 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b7c3b-5ba5-4809-bb91-7eb25cc2a400 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:45.913296 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b7c3b-5ba5-4809-bb91-7eb25cc2a400 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:45.913296 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b7c3b-5ba5-4809-bb91-7eb25cc2a400 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:45.913296 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 20/40] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:13:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:13:45.917046 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b7c3b-5ba5-4809-bb91-7eb25cc2a400 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:45.995313 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 21/41] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Aug 23 04:13:45 2022] POST /identity/v3/projects => generated 301 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:13:45.996598 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827b7e9-a051-47aa-a828-d101f3f6afb1 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:45.996906 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827b7e9-a051-47aa-a828-d101f3f6afb1 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:45.997175 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827b7e9-a051-47aa-a828-d101f3f6afb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:46.025978 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-4827b7e9-a051-47aa-a828-d101f3f6afb1 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:13:46.027667 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-4827b7e9-a051-47aa-a828-d101f3f6afb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:13:46.050854 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b7c3b-5ba5-4809-bb91-7eb25cc2a400 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:46.094273 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab002602-8b65-4885-baa5-f823185dc616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:46.094955 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab002602-8b65-4885-baa5-f823185dc616 None admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:46.095159 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab002602-8b65-4885-baa5-f823185dc616 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:46.095439 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab002602-8b65-4885-baa5-f823185dc616 None admin] PATH_INFO: `/v3/domains/default/users/de65eb39f13f4a92a6a526fb5ab1f5c5/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:46.097688 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab002602-8b65-4885-baa5-f823185dc616 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=de65eb39f13f4a92a6a526fb5ab1f5c5, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:46.099192 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab002602-8b65-4885-baa5-f823185dc616 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:46.105761 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 22/42] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:13:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:13:46.106804 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827b7e9-a051-47aa-a828-d101f3f6afb1 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:46.128945 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 21/43] 10.222.0.18 () {58 vars in 1361 bytes} [Tue Aug 23 04:13:45 2022] PUT /identity/v3/domains/default/users/de65eb39f13f4a92a6a526fb5ab1f5c5/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:13:46.155565 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:46.239786 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827b7e9-a051-47aa-a828-d101f3f6afb1 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:46.279453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f146990-388a-4765-8161-448eafa53bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:46.280124 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f146990-388a-4765-8161-448eafa53bdc None admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:46.280407 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f146990-388a-4765-8161-448eafa53bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:46.280679 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f146990-388a-4765-8161-448eafa53bdc None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:46.281349 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f146990-388a-4765-8161-448eafa53bdc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:46.282888 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f146990-388a-4765-8161-448eafa53bdc None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:46.299492 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 23/44] 10.222.0.18 () {60 vars in 1131 bytes} [Tue Aug 23 04:13:46 2022] POST /identity/v3/roles => generated 223 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:13:46.371670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab002602-8b65-4885-baa5-f823185dc616 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:46.382198 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:46.483145 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab002602-8b65-4885-baa5-f823185dc616 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:46.520410 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590eb77-ddaa-4e99-95e0-86b0cb5e3f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:46.521058 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590eb77-ddaa-4e99-95e0-86b0cb5e3f63 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:46.521332 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590eb77-ddaa-4e99-95e0-86b0cb5e3f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:46.521604 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590eb77-ddaa-4e99-95e0-86b0cb5e3f63 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:46.522687 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5590eb77-ddaa-4e99-95e0-86b0cb5e3f63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:46.524239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5590eb77-ddaa-4e99-95e0-86b0cb5e3f63 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:46.525816 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 22/45] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:13:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:46.531403 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f146990-388a-4765-8161-448eafa53bdc None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:46.628361 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:46.635176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f146990-388a-4765-8161-448eafa53bdc None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:46.669191 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d949d-77ce-4c6b-93dd-a589976fec65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:46.669810 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d949d-77ce-4c6b-93dd-a589976fec65 None admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:46.670090 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d949d-77ce-4c6b-93dd-a589976fec65 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:46.670359 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d949d-77ce-4c6b-93dd-a589976fec65 None admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:46.670992 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d949d-77ce-4c6b-93dd-a589976fec65 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:46.672582 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d949d-77ce-4c6b-93dd-a589976fec65 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:46.700315 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 24/46] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Aug 23 04:13:46 2022] POST /identity/v3/projects => generated 312 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:13:48.051998 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb85538-8330-4e1b-86c8-f27bd8006b8f None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:48.052538 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb85538-8330-4e1b-86c8-f27bd8006b8f None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:48.052538 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb85538-8330-4e1b-86c8-f27bd8006b8f None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:48.053943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 23/47] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:13:48.059476 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d46719-580f-4e82-8cf4-8c75b5bf3ece None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:48.059886 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d46719-580f-4e82-8cf4-8c75b5bf3ece None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:48.060059 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d46719-580f-4e82-8cf4-8c75b5bf3ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:48.078666 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-82d46719-580f-4e82-8cf4-8c75b5bf3ece None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:13:48.079863 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-82d46719-580f-4e82-8cf4-8c75b5bf3ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:13:48.149436 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 25/48] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:13:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:13:48.410400 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb85538-8330-4e1b-86c8-f27bd8006b8f None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:48.494653 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:48.520340 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb85538-8330-4e1b-86c8-f27bd8006b8f None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:48.556314 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5fc93f-7936-4fcf-b4e2-0b4cc6515e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:48.556954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5fc93f-7936-4fcf-b4e2-0b4cc6515e93 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:48.557234 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5fc93f-7936-4fcf-b4e2-0b4cc6515e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:48.557506 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5fc93f-7936-4fcf-b4e2-0b4cc6515e93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:48.567488 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5fc93f-7936-4fcf-b4e2-0b4cc6515e93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:13:48.572351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-ca5fc93f-7936-4fcf-b4e2-0b4cc6515e93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:13:48.572351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:13:48.572351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:13:48.572351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:13:48.572351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:13:48.572351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:13:48.572351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:13:48.572351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:13:48.572351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:13:48.572351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:13:48.572351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:13:48.572351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:13:48.572351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:13:48.572351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:13:48.572351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:13:48.572351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:13:48.573620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:13:48.573620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:13:48.573620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:13:48.573620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:13:48.573620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:13:48.573620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:13:48.573620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:13:48.573620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:13:48.573620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:13:48.573620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:13:48.573620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:13:48.573620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:13:48.573620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:13:48.573620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:13:48.573620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:13:48.573620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:13:48.574732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:13:48.574732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:13:48.574732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:13:48.574732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:13:48.574732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:13:48.574732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:13:48.574732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:13:48.574732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:13:48.574732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:13:48.574732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:13:48.574732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:13:48.574732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:13:48.574732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 24/49] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:13:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:13:48.575395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4808d-15ea-4544-808e-ff7e5f0b3cbc None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:48.575395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4808d-15ea-4544-808e-ff7e5f0b3cbc None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:48.575395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4808d-15ea-4544-808e-ff7e5f0b3cbc None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:48.576554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 26/50] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:13:48.579280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5fc93f-7936-4fcf-b4e2-0b4cc6515e93 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:48.581761 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770a91a-e0f9-498b-8a36-ed798ed992e6 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:48.582051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770a91a-e0f9-498b-8a36-ed798ed992e6 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:48.582331 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770a91a-e0f9-498b-8a36-ed798ed992e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:48.602618 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-6770a91a-e0f9-498b-8a36-ed798ed992e6 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:13:48.603847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-6770a91a-e0f9-498b-8a36-ed798ed992e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:13:48.671459 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 27/51] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:13:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:13:48.683545 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5fc93f-7936-4fcf-b4e2-0b4cc6515e93 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:48.720403 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d585e4c-66aa-42c0-90a8-767c4c6e093c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:48.721019 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d585e4c-66aa-42c0-90a8-767c4c6e093c None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:48.721297 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d585e4c-66aa-42c0-90a8-767c4c6e093c None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:48.721572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d585e4c-66aa-42c0-90a8-767c4c6e093c None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:48.722240 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d585e4c-66aa-42c0-90a8-767c4c6e093c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:13:48.722540 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d585e4c-66aa-42c0-90a8-767c4c6e093c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:48.723944 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d585e4c-66aa-42c0-90a8-767c4c6e093c None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:48.725468 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:48.737323 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 25/52] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:13:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:48.743268 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770a91a-e0f9-498b-8a36-ed798ed992e6 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:48.756053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:48.849694 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770a91a-e0f9-498b-8a36-ed798ed992e6 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:48.885910 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea8018-c030-48b5-a99a-f3e10711c113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:48.886548 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea8018-c030-48b5-a99a-f3e10711c113 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:48.886833 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea8018-c030-48b5-a99a-f3e10711c113 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:48.887114 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea8018-c030-48b5-a99a-f3e10711c113 None admin] PATH_INFO: `/v3/users/de65eb39f13f4a92a6a526fb5ab1f5c5` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:48.888304 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea8018-c030-48b5-a99a-f3e10711c113 None admin] RBAC: Authorizing `identity:get_user(user_id=de65eb39f13f4a92a6a526fb5ab1f5c5)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:48.889772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea8018-c030-48b5-a99a-f3e10711c113 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:48.891366 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 28/53] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:13:48 2022] GET /identity/v3/users/de65eb39f13f4a92a6a526fb5ab1f5c5 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:48.897284 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d585e4c-66aa-42c0-90a8-767c4c6e093c None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:48.949910 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea8018-c030-48b5-a99a-f3e10711c113 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:49.003248 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d585e4c-66aa-42c0-90a8-767c4c6e093c None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:49.032907 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:49.042766 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd578d3c-5f9f-4aba-b0fc-9c7105261472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:49.043413 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd578d3c-5f9f-4aba-b0fc-9c7105261472 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:49.043697 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd578d3c-5f9f-4aba-b0fc-9c7105261472 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:49.043993 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd578d3c-5f9f-4aba-b0fc-9c7105261472 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:49.045207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd578d3c-5f9f-4aba-b0fc-9c7105261472 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:49.046686 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd578d3c-5f9f-4aba-b0fc-9c7105261472 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:49.048318 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 26/54] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:13:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:49.054177 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd578d3c-5f9f-4aba-b0fc-9c7105261472 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:49.056346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea8018-c030-48b5-a99a-f3e10711c113 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:49.099266 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461adfe-c083-4d04-8a51-df7aae94d1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:49.099972 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461adfe-c083-4d04-8a51-df7aae94d1f0 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:49.100215 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461adfe-c083-4d04-8a51-df7aae94d1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:49.100498 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461adfe-c083-4d04-8a51-df7aae94d1f0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:49.101728 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461adfe-c083-4d04-8a51-df7aae94d1f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:49.103277 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461adfe-c083-4d04-8a51-df7aae94d1f0 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:49.104916 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 29/55] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:13:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:49.110322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461adfe-c083-4d04-8a51-df7aae94d1f0 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:49.117658 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:49.157358 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd578d3c-5f9f-4aba-b0fc-9c7105261472 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:49.192154 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ecf95-0d2d-4f8f-ab68-fbab53dd9ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:49.192751 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ecf95-0d2d-4f8f-ab68-fbab53dd9ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:49.193031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ecf95-0d2d-4f8f-ab68-fbab53dd9ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:49.193303 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ecf95-0d2d-4f8f-ab68-fbab53dd9ad5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:49.194032 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ecf95-0d2d-4f8f-ab68-fbab53dd9ad5 None admin] RBAC: Adding query filter params (role.id=52b31061b72c45ccb76dd26bf31f8d3f, scope.domain.id=default, user.id=de65eb39f13f4a92a6a526fb5ab1f5c5) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:13:49.194336 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ecf95-0d2d-4f8f-ab68-fbab53dd9ad5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:49.195795 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ecf95-0d2d-4f8f-ab68-fbab53dd9ad5 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:49.205825 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 27/56] 10.222.0.18 () {56 vars in 1322 bytes} [Tue Aug 23 04:13:49 2022] GET /identity/v3/role_assignments?user.id=de65eb39f13f4a92a6a526fb5ab1f5c5&scope.domain.id=default&role.id=52b31061b72c45ccb76dd26bf31f8d3f => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:49.212229 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461adfe-c083-4d04-8a51-df7aae94d1f0 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:49.251256 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfd1d38-dc75-48e7-ae47-72af49573934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:49.251939 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfd1d38-dc75-48e7-ae47-72af49573934 None admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:49.252227 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfd1d38-dc75-48e7-ae47-72af49573934 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:49.252506 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfd1d38-dc75-48e7-ae47-72af49573934 None admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:49.253175 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfd1d38-dc75-48e7-ae47-72af49573934 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:49.254720 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfd1d38-dc75-48e7-ae47-72af49573934 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:49.272877 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:49.287247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 30/57] 10.222.0.18 () {60 vars in 1140 bytes} [Tue Aug 23 04:13:49 2022] POST /identity/v3/projects => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:13:51.195483 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6743dd-4fe7-4d3d-964f-434e47a7fa02 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:51.196072 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6743dd-4fe7-4d3d-964f-434e47a7fa02 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:51.196072 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6743dd-4fe7-4d3d-964f-434e47a7fa02 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:51.197468 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 28/58] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:13:51.203836 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d6283-fbdd-4976-9b2c-ba4482d9e3fb None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:51.204534 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d6283-fbdd-4976-9b2c-ba4482d9e3fb None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:51.204800 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d6283-fbdd-4976-9b2c-ba4482d9e3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:51.229719 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-6c9d6283-fbdd-4976-9b2c-ba4482d9e3fb None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:13:51.230912 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-6c9d6283-fbdd-4976-9b2c-ba4482d9e3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:13:51.302998 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 31/59] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:13:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:13:51.568959 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6743dd-4fe7-4d3d-964f-434e47a7fa02 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:51.691132 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6743dd-4fe7-4d3d-964f-434e47a7fa02 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:51.717888 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:51.728005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a215ed-6f05-4951-bf88-b9d4a8a4eadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:51.729028 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a215ed-6f05-4951-bf88-b9d4a8a4eadb None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:51.729536 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a215ed-6f05-4951-bf88-b9d4a8a4eadb None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:51.729801 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a215ed-6f05-4951-bf88-b9d4a8a4eadb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:51.731030 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a215ed-6f05-4951-bf88-b9d4a8a4eadb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:51.732529 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a215ed-6f05-4951-bf88-b9d4a8a4eadb None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:51.734097 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 29/60] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:13:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:51.740691 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d6283-fbdd-4976-9b2c-ba4482d9e3fb None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:51.865327 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d6283-fbdd-4976-9b2c-ba4482d9e3fb None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:51.892839 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:51.902690 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270387c4-f35b-4c26-9ee9-ce0cf23d96f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:51.903329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270387c4-f35b-4c26-9ee9-ce0cf23d96f1 None admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:51.903612 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270387c4-f35b-4c26-9ee9-ce0cf23d96f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:51.903886 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270387c4-f35b-4c26-9ee9-ce0cf23d96f1 None admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:51.904571 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270387c4-f35b-4c26-9ee9-ce0cf23d96f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:51.906026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270387c4-f35b-4c26-9ee9-ce0cf23d96f1 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:51.934402 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 32/61] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Aug 23 04:13:51 2022] POST /identity/v3/users => generated 275 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:13:54.893210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57873127-7274-49aa-b7cf-79615eb93054 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:54.894223 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57873127-7274-49aa-b7cf-79615eb93054 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:54.894694 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57873127-7274-49aa-b7cf-79615eb93054 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:54.896299 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 30/62] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:13:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:13:54.902092 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9ae42-363e-4770-abca-92ea81c1c6ba None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:54.902972 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9ae42-363e-4770-abca-92ea81c1c6ba None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:54.903446 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9ae42-363e-4770-abca-92ea81c1c6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:54.936198 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-7cb9ae42-363e-4770-abca-92ea81c1c6ba None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:13:54.937405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-7cb9ae42-363e-4770-abca-92ea81c1c6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:13:54.952635 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbdfdc0-137b-4bca-bb1c-a617909089cb None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:54.953606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbdfdc0-137b-4bca-bb1c-a617909089cb None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:54.954142 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbdfdc0-137b-4bca-bb1c-a617909089cb None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:54.956574 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 31/63] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:13:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:13:54.970466 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcba6b7-6560-4aaa-8c98-3189c8dbc46d None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:54.971029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcba6b7-6560-4aaa-8c98-3189c8dbc46d None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:54.971494 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcba6b7-6560-4aaa-8c98-3189c8dbc46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:54.997906 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-9dcba6b7-6560-4aaa-8c98-3189c8dbc46d None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:13:55.000772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-9dcba6b7-6560-4aaa-8c98-3189c8dbc46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:13:55.058066 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 33/64] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:13:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:13:55.105009 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 32/65] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:13:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 04:13:55.263011 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08ef2b-b81b-4b84-98c6-fd8b95edd2cb None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:55.263729 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08ef2b-b81b-4b84-98c6-fd8b95edd2cb None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:55.264296 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08ef2b-b81b-4b84-98c6-fd8b95edd2cb None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:55.266730 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 34/66] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:13:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:13:55.272401 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ea1ea-b020-46fa-a20a-128866218c1a None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:55.272911 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ea1ea-b020-46fa-a20a-128866218c1a None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:55.273395 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ea1ea-b020-46fa-a20a-128866218c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:55.295969 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-c51ea1ea-b020-46fa-a20a-128866218c1a None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:13:55.297495 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-c51ea1ea-b020-46fa-a20a-128866218c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:13:55.381142 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 33/67] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:13:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:13:55.471506 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08ef2b-b81b-4b84-98c6-fd8b95edd2cb None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:55.513985 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ea1ea-b020-46fa-a20a-128866218c1a None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:55.596998 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:55.617523 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08ef2b-b81b-4b84-98c6-fd8b95edd2cb None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:55.631405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:55.645327 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ea1ea-b020-46fa-a20a-128866218c1a None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:55.654264 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf42253-da9f-4251-a7d9-045be622aa79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:55.654912 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf42253-da9f-4251-a7d9-045be622aa79 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:55.655201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf42253-da9f-4251-a7d9-045be622aa79 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:55.655472 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf42253-da9f-4251-a7d9-045be622aa79 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:55.666031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf42253-da9f-4251-a7d9-045be622aa79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:13:55.671000 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-2cf42253-da9f-4251-a7d9-045be622aa79 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:13:55.671000 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:13:55.671000 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:13:55.671000 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:13:55.671000 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:13:55.671000 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:13:55.671000 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:13:55.671000 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:13:55.671000 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:13:55.671000 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:13:55.671000 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:13:55.671000 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:13:55.671000 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:13:55.671000 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:13:55.671000 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:13:55.671000 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:13:55.671896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:13:55.671896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:13:55.671896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:13:55.671896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:13:55.671896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:13:55.671896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:13:55.671896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:13:55.671896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:13:55.671896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:13:55.671896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:13:55.671896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:13:55.671896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:13:55.671896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:13:55.671896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:13:55.671896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:13:55.671896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:13:55.676792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:13:55.676792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:13:55.676792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:13:55.676792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:13:55.676792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:13:55.676792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:13:55.676792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:13:55.676792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:13:55.676792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:13:55.676792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:13:55.676792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:13:55.676792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:13:55.676792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 35/68] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:13:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:13:55.676792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba177f22-f92c-46c2-a5f5-c6248a971856 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:55.677452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba177f22-f92c-46c2-a5f5-c6248a971856 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:55.677452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba177f22-f92c-46c2-a5f5-c6248a971856 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:55.677452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 36/69] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:13:55.677452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212164e0-be7d-4058-9dbb-ddc5f714f300 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:55.677452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212164e0-be7d-4058-9dbb-ddc5f714f300 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:55.677452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212164e0-be7d-4058-9dbb-ddc5f714f300 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:55.682889 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 37/70] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:13:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:13:55.683572 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212164e0-be7d-4058-9dbb-ddc5f714f300 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:55.688918 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a99ef-33a9-46e5-9dd5-bc9410d1d1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:55.689561 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a99ef-33a9-46e5-9dd5-bc9410d1d1fb None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:55.689838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a99ef-33a9-46e5-9dd5-bc9410d1d1fb None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:55.690106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a99ef-33a9-46e5-9dd5-bc9410d1d1fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:55.701930 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a99ef-33a9-46e5-9dd5-bc9410d1d1fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:13:55.707455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-903a99ef-33a9-46e5-9dd5-bc9410d1d1fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:13:55.707455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:13:55.707455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:13:55.707455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:13:55.707455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:13:55.707455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:13:55.707455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:13:55.707455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:13:55.707455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:13:55.707455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:13:55.707455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:13:55.707455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:13:55.707455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:13:55.707455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:13:55.707455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:13:55.707455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:13:55.708492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:13:55.708492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:13:55.708492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:13:55.708492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:13:55.708492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:13:55.708492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:13:55.708492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:13:55.708492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:13:55.708492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:13:55.708492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:13:55.708492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:13:55.708492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:13:55.708492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:13:55.708492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:13:55.708492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:13:55.708492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:13:55.712861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:13:55.712861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:13:55.712861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:13:55.712861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:13:55.712861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:13:55.712861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:13:55.712861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:13:55.712861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:13:55.712861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:13:55.712861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:13:55.712861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:13:55.712861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:13:55.712861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 34/71] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:13:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:13:55.712861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb8393-8763-492e-94f2-4b6b007f6a67 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:55.713509 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb8393-8763-492e-94f2-4b6b007f6a67 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:55.713509 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb8393-8763-492e-94f2-4b6b007f6a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:55.728571 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-25cb8393-8763-492e-94f2-4b6b007f6a67 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:13:55.729757 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-25cb8393-8763-492e-94f2-4b6b007f6a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:13:55.792125 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212164e0-be7d-4058-9dbb-ddc5f714f300 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:55.798546 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 35/72] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:13:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:13:55.800216 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af7ec3-01f1-4fd0-9e1d-edbf0e7bd349 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:55.800512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af7ec3-01f1-4fd0-9e1d-edbf0e7bd349 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:55.800787 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af7ec3-01f1-4fd0-9e1d-edbf0e7bd349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:55.818929 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-63af7ec3-01f1-4fd0-9e1d-edbf0e7bd349 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:13:55.820196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-63af7ec3-01f1-4fd0-9e1d-edbf0e7bd349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:13:55.829783 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973e772-6a94-4032-9328-f5ff4bb83787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:55.830731 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973e772-6a94-4032-9328-f5ff4bb83787 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:55.831013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973e772-6a94-4032-9328-f5ff4bb83787 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:55.831444 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973e772-6a94-4032-9328-f5ff4bb83787 None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:55.832168 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973e772-6a94-4032-9328-f5ff4bb83787 None admin] RBAC: Adding query filter params (name=member) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:13:55.832556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973e772-6a94-4032-9328-f5ff4bb83787 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:55.834164 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973e772-6a94-4032-9328-f5ff4bb83787 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:55.847651 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 38/73] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Aug 23 04:13:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:55.847651 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973e772-6a94-4032-9328-f5ff4bb83787 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:55.872638 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:55.879798 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:55.900835 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 36/74] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:13:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:13:55.901759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af7ec3-01f1-4fd0-9e1d-edbf0e7bd349 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:55.954628 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973e772-6a94-4032-9328-f5ff4bb83787 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:55.991845 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd090ad3-46bc-4272-a176-452989f7084c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:55.992547 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd090ad3-46bc-4272-a176-452989f7084c None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:55.992991 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd090ad3-46bc-4272-a176-452989f7084c None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:55.993278 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd090ad3-46bc-4272-a176-452989f7084c None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:55.993961 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd090ad3-46bc-4272-a176-452989f7084c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:13:55.994270 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd090ad3-46bc-4272-a176-452989f7084c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:55.995781 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd090ad3-46bc-4272-a176-452989f7084c None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:56.005064 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af7ec3-01f1-4fd0-9e1d-edbf0e7bd349 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:56.005209 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 39/75] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:13:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:56.010480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd090ad3-46bc-4272-a176-452989f7084c None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:56.121589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:56.128305 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd090ad3-46bc-4272-a176-452989f7084c None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:56.157741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df016d-6134-4d6f-85e7-f9d3594efd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:56.158381 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df016d-6134-4d6f-85e7-f9d3594efd08 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:56.158663 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df016d-6134-4d6f-85e7-f9d3594efd08 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:56.158929 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df016d-6134-4d6f-85e7-f9d3594efd08 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:56.166006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39814b37-6b1d-4c76-b6fe-b8b7f7afbc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:56.166657 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39814b37-6b1d-4c76-b6fe-b8b7f7afbc56 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:56.167101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39814b37-6b1d-4c76-b6fe-b8b7f7afbc56 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:56.167385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39814b37-6b1d-4c76-b6fe-b8b7f7afbc56 None admin] PATH_INFO: `/v3/users/628f03c8b9b44e27bacc6919029346d1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:56.171238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63df016d-6134-4d6f-85e7-f9d3594efd08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:13:56.176315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-63df016d-6134-4d6f-85e7-f9d3594efd08 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 04:13:56.176315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:13:56.176315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:13:56.176315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:13:56.176315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:13:56.176315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:13:56.176315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:13:56.176315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:13:56.176315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:13:56.176315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:13:56.176315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:13:56.176315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:13:56.176315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:13:56.176315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:13:56.176315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:13:56.176315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:13:56.177159 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:13:56.177159 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:13:56.177159 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:13:56.177159 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:13:56.177159 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:13:56.177159 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:13:56.177159 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:13:56.177159 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:13:56.177159 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:13:56.177159 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:13:56.177159 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:13:56.177159 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:13:56.177159 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:13:56.177159 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:13:56.177159 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:13:56.177159 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:13:56.179551 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:13:56.179551 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:13:56.179551 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:13:56.179551 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:13:56.179551 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:13:56.179551 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:13:56.179551 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:13:56.179551 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:13:56.179551 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:13:56.179551 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:13:56.179551 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 04:13:56.179551 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:13:56.179551 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 37/76] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Aug 23 04:13:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:13:56.179551 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df016d-6134-4d6f-85e7-f9d3594efd08 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:56.184593 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39814b37-6b1d-4c76-b6fe-b8b7f7afbc56 None admin] RBAC: Authorizing `identity:get_user(user_id=628f03c8b9b44e27bacc6919029346d1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:56.186382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39814b37-6b1d-4c76-b6fe-b8b7f7afbc56 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:56.188137 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 40/77] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:13:56 2022] GET /identity/v3/users/628f03c8b9b44e27bacc6919029346d1 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:56.188908 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39814b37-6b1d-4c76-b6fe-b8b7f7afbc56 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:56.217774 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:56.281448 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df016d-6134-4d6f-85e7-f9d3594efd08 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:56.285647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39814b37-6b1d-4c76-b6fe-b8b7f7afbc56 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:56.315604 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6157d-2fd5-44ff-a49c-eae45c8472bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:56.316307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6157d-2fd5-44ff-a49c-eae45c8472bc None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:56.316592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6157d-2fd5-44ff-a49c-eae45c8472bc None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:56.316865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6157d-2fd5-44ff-a49c-eae45c8472bc None admin] PATH_INFO: `/v3/users/de65eb39f13f4a92a6a526fb5ab1f5c5` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:56.318021 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e6157d-2fd5-44ff-a49c-eae45c8472bc None admin] RBAC: Authorizing `identity:get_user(user_id=de65eb39f13f4a92a6a526fb5ab1f5c5)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:56.319544 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e6157d-2fd5-44ff-a49c-eae45c8472bc None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:56.320315 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7f52e-af4e-45ce-919d-8e349e9fe29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:56.321070 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7f52e-af4e-45ce-919d-8e349e9fe29d None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:56.321332 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 38/78] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:13:56 2022] GET /identity/v3/users/de65eb39f13f4a92a6a526fb5ab1f5c5 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:56.321441 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7f52e-af4e-45ce-919d-8e349e9fe29d None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:56.321595 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7f52e-af4e-45ce-919d-8e349e9fe29d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:56.322777 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e7f52e-af4e-45ce-919d-8e349e9fe29d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:56.324241 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e7f52e-af4e-45ce-919d-8e349e9fe29d None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:56.324873 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6157d-2fd5-44ff-a49c-eae45c8472bc None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:56.325857 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 41/79] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:13:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:56.328036 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7f52e-af4e-45ce-919d-8e349e9fe29d None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:56.353928 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:56.426139 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6157d-2fd5-44ff-a49c-eae45c8472bc None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:56.428416 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:56.430609 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7f52e-af4e-45ce-919d-8e349e9fe29d None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:56.464543 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af333c7-eee8-45a6-8136-3f46d71bd5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:56.465160 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af333c7-eee8-45a6-8136-3f46d71bd5a7 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:56.465439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af333c7-eee8-45a6-8136-3f46d71bd5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:56.465706 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af333c7-eee8-45a6-8136-3f46d71bd5a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:56.466372 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af333c7-eee8-45a6-8136-3f46d71bd5a7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:13:56.466671 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af333c7-eee8-45a6-8136-3f46d71bd5a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:56.466774 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4fb916-96e0-4aa3-9684-43cf8c759144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:56.467423 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fb916-96e0-4aa3-9684-43cf8c759144 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:56.467704 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fb916-96e0-4aa3-9684-43cf8c759144 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:56.468010 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fb916-96e0-4aa3-9684-43cf8c759144 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:56.468097 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af333c7-eee8-45a6-8136-3f46d71bd5a7 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:56.476157 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 42/80] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:13:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:56.476942 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af333c7-eee8-45a6-8136-3f46d71bd5a7 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:56.479245 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4fb916-96e0-4aa3-9684-43cf8c759144 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:13:56.484035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-0d4fb916-96e0-4aa3-9684-43cf8c759144 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:13:56.484035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:13:56.484035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:13:56.484035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:13:56.484035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:13:56.484035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:13:56.484035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:13:56.484035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:13:56.484035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:13:56.484035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:13:56.484035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:13:56.484035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:13:56.484035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:13:56.484035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:13:56.484035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:13:56.484035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:13:56.484854 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:13:56.484854 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:13:56.484854 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:13:56.484854 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:13:56.484854 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:13:56.484854 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:13:56.484854 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:13:56.484854 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:13:56.484854 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:13:56.484854 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:13:56.484854 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:13:56.484854 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:13:56.484854 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:13:56.484854 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:13:56.484854 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:13:56.484854 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:13:56.485585 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:13:56.485585 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:13:56.485585 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:13:56.485585 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:13:56.485585 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:13:56.485585 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:13:56.485585 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:13:56.485585 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:13:56.485585 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:13:56.485585 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:13:56.485585 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:13:56.485585 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:13:56.485585 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 39/81] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:13:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:13:56.486178 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4fb916-96e0-4aa3-9684-43cf8c759144 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:56.561596 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:56.578182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af333c7-eee8-45a6-8136-3f46d71bd5a7 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:56.589864 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4fb916-96e0-4aa3-9684-43cf8c759144 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:56.612531 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab064f75-bb5d-4db5-b351-ecffaccae368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:56.613154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab064f75-bb5d-4db5-b351-ecffaccae368 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:56.613433 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab064f75-bb5d-4db5-b351-ecffaccae368 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:56.613709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab064f75-bb5d-4db5-b351-ecffaccae368 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:56.623635 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4bc976-cd56-4ab1-b514-d60b8d01cb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:56.624282 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bc976-cd56-4ab1-b514-d60b8d01cb63 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:56.624562 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bc976-cd56-4ab1-b514-d60b8d01cb63 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:56.624835 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bc976-cd56-4ab1-b514-d60b8d01cb63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:56.656694 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:56.678591 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab064f75-bb5d-4db5-b351-ecffaccae368 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:56.680255 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab064f75-bb5d-4db5-b351-ecffaccae368 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:56.693475 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4bc976-cd56-4ab1-b514-d60b8d01cb63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:56.695066 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4bc976-cd56-4ab1-b514-d60b8d01cb63 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:56.763086 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:56.779878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 43/82] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:13:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:13:56.780592 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab064f75-bb5d-4db5-b351-ecffaccae368 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:56.794309 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 40/83] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:13:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:13:56.796885 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4bc976-cd56-4ab1-b514-d60b8d01cb63 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:56.857788 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:56.884891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab064f75-bb5d-4db5-b351-ecffaccae368 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:56.904240 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4bc976-cd56-4ab1-b514-d60b8d01cb63 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:56.922855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75ba19-b5c9-4561-885a-5479cb7cb15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:56.923474 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75ba19-b5c9-4561-885a-5479cb7cb15c None admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:56.923751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75ba19-b5c9-4561-885a-5479cb7cb15c None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:56.924055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75ba19-b5c9-4561-885a-5479cb7cb15c None admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:56.924697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75ba19-b5c9-4561-885a-5479cb7cb15c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:56.926181 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75ba19-b5c9-4561-885a-5479cb7cb15c None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:56.941061 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579274c2-4217-4f65-b94a-eaa0fa916d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:56.941776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579274c2-4217-4f65-b94a-eaa0fa916d57 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:56.942087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579274c2-4217-4f65-b94a-eaa0fa916d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:56.942366 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579274c2-4217-4f65-b94a-eaa0fa916d57 None admin] PATH_INFO: `/v3/users/628f03c8b9b44e27bacc6919029346d1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:56.943514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579274c2-4217-4f65-b94a-eaa0fa916d57 None admin] RBAC: Authorizing `identity:get_user(user_id=628f03c8b9b44e27bacc6919029346d1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:56.945181 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579274c2-4217-4f65-b94a-eaa0fa916d57 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:56.947097 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 41/84] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:13:56 2022] GET /identity/v3/users/628f03c8b9b44e27bacc6919029346d1 => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:56.947858 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579274c2-4217-4f65-b94a-eaa0fa916d57 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:56.960326 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 44/85] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Aug 23 04:13:56 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:13:56.960326 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75ba19-b5c9-4561-885a-5479cb7cb15c None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:57.007837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:57.067390 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579274c2-4217-4f65-b94a-eaa0fa916d57 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:57.077537 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75ba19-b5c9-4561-885a-5479cb7cb15c None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:57.105519 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:57.112743 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d8486-a72e-456e-94bf-2d0802afff80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:57.113682 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d8486-a72e-456e-94bf-2d0802afff80 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:57.113969 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d8486-a72e-456e-94bf-2d0802afff80 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:57.114245 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d8486-a72e-456e-94bf-2d0802afff80 None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:57.114928 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479d8486-a72e-456e-94bf-2d0802afff80 None admin] RBAC: Adding query filter params (name=member) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:13:57.115229 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479d8486-a72e-456e-94bf-2d0802afff80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:57.116649 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479d8486-a72e-456e-94bf-2d0802afff80 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:57.118480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964069d-dd43-4fc6-82f1-63cf8b218a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:57.119481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964069d-dd43-4fc6-82f1-63cf8b218a28 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:57.119755 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964069d-dd43-4fc6-82f1-63cf8b218a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:57.120117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964069d-dd43-4fc6-82f1-63cf8b218a28 None admin] PATH_INFO: `/v3/projects/e3148f82c47e4702b119cf867f5dcdcc` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:57.121275 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5964069d-dd43-4fc6-82f1-63cf8b218a28 None admin] RBAC: Authorizing `identity:get_project(project_id=e3148f82c47e4702b119cf867f5dcdcc)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:57.122732 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5964069d-dd43-4fc6-82f1-63cf8b218a28 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:57.126636 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 45/86] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:13:56 2022] GET /identity/v3/projects/e3148f82c47e4702b119cf867f5dcdcc => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:57.126636 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964069d-dd43-4fc6-82f1-63cf8b218a28 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:57.126814 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 42/87] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Aug 23 04:13:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:57.126814 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d8486-a72e-456e-94bf-2d0802afff80 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:57.253934 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964069d-dd43-4fc6-82f1-63cf8b218a28 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:57.264004 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d8486-a72e-456e-94bf-2d0802afff80 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:57.264245 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:57.301372 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5665f1-947e-4439-8614-e67b350e325a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:57.302384 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5665f1-947e-4439-8614-e67b350e325a None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:57.302792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5665f1-947e-4439-8614-e67b350e325a None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:57.303236 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5665f1-947e-4439-8614-e67b350e325a None admin] PATH_INFO: `/v3/projects/e3148f82c47e4702b119cf867f5dcdcc` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:57.304820 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5665f1-947e-4439-8614-e67b350e325a None admin] RBAC: Authorizing `identity:get_project(project_id=e3148f82c47e4702b119cf867f5dcdcc)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:57.305920 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5665f1-947e-4439-8614-e67b350e325a None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:57.307577 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 46/88] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:13:57 2022] GET /identity/v3/projects/e3148f82c47e4702b119cf867f5dcdcc => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:57.308535 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d8159-76de-423f-b0e0-3722c20801fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:57.309422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d8159-76de-423f-b0e0-3722c20801fc None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:57.309897 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d8159-76de-423f-b0e0-3722c20801fc None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:57.310362 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d8159-76de-423f-b0e0-3722c20801fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:57.312832 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5665f1-947e-4439-8614-e67b350e325a None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:57.385842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:57.398606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d8159-76de-423f-b0e0-3722c20801fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:57.401805 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d8159-76de-423f-b0e0-3722c20801fc None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:57.444890 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5665f1-947e-4439-8614-e67b350e325a None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:57.486507 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64695866-df31-4cb6-a9da-991d95e9b00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:57.487621 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64695866-df31-4cb6-a9da-991d95e9b00e None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:57.488214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64695866-df31-4cb6-a9da-991d95e9b00e None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:57.488676 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64695866-df31-4cb6-a9da-991d95e9b00e None admin] PATH_INFO: `/v3/users/628f03c8b9b44e27bacc6919029346d1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:57.490013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64695866-df31-4cb6-a9da-991d95e9b00e None admin] RBAC: Authorizing `identity:get_user(user_id=628f03c8b9b44e27bacc6919029346d1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:57.491765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64695866-df31-4cb6-a9da-991d95e9b00e None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:57.493926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 47/89] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:13:57 2022] GET /identity/v3/users/628f03c8b9b44e27bacc6919029346d1 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:57.494912 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64695866-df31-4cb6-a9da-991d95e9b00e None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:57.543788 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 43/90] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:13:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:13:57.545073 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d8159-76de-423f-b0e0-3722c20801fc None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:57.552011 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:57.613051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64695866-df31-4cb6-a9da-991d95e9b00e None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:57.649757 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19eed20-7d76-443d-9c76-3eb059990df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:57.650556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19eed20-7d76-443d-9c76-3eb059990df6 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:57.650731 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19eed20-7d76-443d-9c76-3eb059990df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:57.650983 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19eed20-7d76-443d-9c76-3eb059990df6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:57.651718 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19eed20-7d76-443d-9c76-3eb059990df6 None admin] RBAC: Adding query filter params (role.id=aa03504b4fb642d288d07f4335e291e1, scope.project.id=e3148f82c47e4702b119cf867f5dcdcc, user.id=628f03c8b9b44e27bacc6919029346d1) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:13:57.652156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19eed20-7d76-443d-9c76-3eb059990df6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:57.653581 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19eed20-7d76-443d-9c76-3eb059990df6 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:57.664076 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 48/91] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:13:57 2022] GET /identity/v3/role_assignments?user.id=628f03c8b9b44e27bacc6919029346d1&scope.project.id=e3148f82c47e4702b119cf867f5dcdcc&role.id=aa03504b4fb642d288d07f4335e291e1 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:57.664076 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19eed20-7d76-443d-9c76-3eb059990df6 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:57.676459 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:57.693171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d8159-76de-423f-b0e0-3722c20801fc None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:57.815487 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d68dbe-546c-43fa-9780-a811ba122022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:57.815487 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d68dbe-546c-43fa-9780-a811ba122022 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:57.815487 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d68dbe-546c-43fa-9780-a811ba122022 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:57.815776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d68dbe-546c-43fa-9780-a811ba122022 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:57.816474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d68dbe-546c-43fa-9780-a811ba122022 None admin] RBAC: Adding query filter params (role.id=52b31061b72c45ccb76dd26bf31f8d3f, scope.project.id=e3148f82c47e4702b119cf867f5dcdcc, user.id=de65eb39f13f4a92a6a526fb5ab1f5c5) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:13:57.816782 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d68dbe-546c-43fa-9780-a811ba122022 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:57.817682 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19eed20-7d76-443d-9c76-3eb059990df6 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:57.818257 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d68dbe-546c-43fa-9780-a811ba122022 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:57.827700 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 44/92] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:13:57 2022] GET /identity/v3/role_assignments?user.id=de65eb39f13f4a92a6a526fb5ab1f5c5&scope.project.id=e3148f82c47e4702b119cf867f5dcdcc&role.id=52b31061b72c45ccb76dd26bf31f8d3f => generated 259 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:57.828609 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d68dbe-546c-43fa-9780-a811ba122022 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:57.838430 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:57.859929 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d318b-147a-4159-b1da-73975845cfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:57.860568 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d318b-147a-4159-b1da-73975845cfdb None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:57.860854 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d318b-147a-4159-b1da-73975845cfdb None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:57.861137 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d318b-147a-4159-b1da-73975845cfdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:57.952112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d318b-147a-4159-b1da-73975845cfdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:57.953712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d318b-147a-4159-b1da-73975845cfdb None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:57.966350 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d68dbe-546c-43fa-9780-a811ba122022 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:58.036724 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485b892-4c7c-4127-86c0-d6d9787778ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:58.037408 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485b892-4c7c-4127-86c0-d6d9787778ba None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:58.037688 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485b892-4c7c-4127-86c0-d6d9787778ba None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:58.037963 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485b892-4c7c-4127-86c0-d6d9787778ba None admin] PATH_INFO: `/v3/projects/e3148f82c47e4702b119cf867f5dcdcc` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:58.044878 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2485b892-4c7c-4127-86c0-d6d9787778ba None admin] RBAC: Authorizing `identity:get_project(project_id=e3148f82c47e4702b119cf867f5dcdcc)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:58.046401 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2485b892-4c7c-4127-86c0-d6d9787778ba None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:58.048519 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 45/93] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:13:57 2022] GET /identity/v3/projects/e3148f82c47e4702b119cf867f5dcdcc => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:58.054567 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485b892-4c7c-4127-86c0-d6d9787778ba None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:58.112306 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:58.122941 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 49/94] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:13:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:13:58.128553 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d318b-147a-4159-b1da-73975845cfdb None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:58.140285 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:58.227379 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485b892-4c7c-4127-86c0-d6d9787778ba None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:58.302749 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d318b-147a-4159-b1da-73975845cfdb None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:58.304614 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d8ea2b-fc0a-46dc-a9b9-bf9c117f87f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:58.305272 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8ea2b-fc0a-46dc-a9b9-bf9c117f87f1 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:58.305555 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8ea2b-fc0a-46dc-a9b9-bf9c117f87f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:58.305831 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8ea2b-fc0a-46dc-a9b9-bf9c117f87f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:58.307020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d8ea2b-fc0a-46dc-a9b9-bf9c117f87f1 None admin] RBAC: Adding query filter params (role.id=4207a743edc04b0aba9197fe3554b2e3, scope.project.id=e3148f82c47e4702b119cf867f5dcdcc, user.id=628f03c8b9b44e27bacc6919029346d1) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:13:58.307020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d8ea2b-fc0a-46dc-a9b9-bf9c117f87f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:58.308356 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d8ea2b-fc0a-46dc-a9b9-bf9c117f87f1 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:58.316509 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 46/95] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:13:58 2022] GET /identity/v3/role_assignments?user.id=628f03c8b9b44e27bacc6919029346d1&scope.project.id=e3148f82c47e4702b119cf867f5dcdcc&role.id=4207a743edc04b0aba9197fe3554b2e3 => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:58.344092 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b1542-4c95-4b10-b47e-f4d791a428dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:58.345141 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b1542-4c95-4b10-b47e-f4d791a428dd None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:58.345622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b1542-4c95-4b10-b47e-f4d791a428dd None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:58.346093 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b1542-4c95-4b10-b47e-f4d791a428dd None admin] PATH_INFO: `/v3/projects/67f78794d6ab4968bc00d61a6015220c` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:58.347634 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3b1542-4c95-4b10-b47e-f4d791a428dd None admin] RBAC: Authorizing `identity:get_project(project_id=67f78794d6ab4968bc00d61a6015220c)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:58.349402 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3b1542-4c95-4b10-b47e-f4d791a428dd None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:58.351452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 50/96] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:13:58 2022] GET /identity/v3/projects/67f78794d6ab4968bc00d61a6015220c => generated 312 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:13:58.357898 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d8ea2b-fc0a-46dc-a9b9-bf9c117f87f1 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:13:58.495497 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d8ea2b-fc0a-46dc-a9b9-bf9c117f87f1 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:13:58.497352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:13:58.532654 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8ee7e-d5d9-4bff-82a3-0ce4dd187576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:13:58.533284 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8ee7e-d5d9-4bff-82a3-0ce4dd187576 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:58.533571 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8ee7e-d5d9-4bff-82a3-0ce4dd187576 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:58.533851 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8ee7e-d5d9-4bff-82a3-0ce4dd187576 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:58.534599 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8ee7e-d5d9-4bff-82a3-0ce4dd187576 None admin] RBAC: Adding query filter params (role.id=aa03504b4fb642d288d07f4335e291e1, scope.project.id=67f78794d6ab4968bc00d61a6015220c, user.id=628f03c8b9b44e27bacc6919029346d1) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:13:58.534916 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8ee7e-d5d9-4bff-82a3-0ce4dd187576 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:13:58.536447 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8ee7e-d5d9-4bff-82a3-0ce4dd187576 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:13:58.543019 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 47/97] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:13:58 2022] GET /identity/v3/role_assignments?user.id=628f03c8b9b44e27bacc6919029346d1&scope.project.id=67f78794d6ab4968bc00d61a6015220c&role.id=aa03504b4fb642d288d07f4335e291e1 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:13:59.450111 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d80aec-47b9-44e1-b95e-4e40304f3323 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:59.450675 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d80aec-47b9-44e1-b95e-4e40304f3323 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:59.450675 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d80aec-47b9-44e1-b95e-4e40304f3323 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:59.454300 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 51/98] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:13:59.460370 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e536f3-e156-4f82-9147-93a5065afa13 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:59.460370 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e536f3-e156-4f82-9147-93a5065afa13 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:59.460370 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e536f3-e156-4f82-9147-93a5065afa13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:59.490101 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-80e536f3-e156-4f82-9147-93a5065afa13 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:13:59.491476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-80e536f3-e156-4f82-9147-93a5065afa13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:13:59.560600 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59739a-ec45-415e-af97-acf0b4d9c2a5 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:59.560933 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59739a-ec45-415e-af97-acf0b4d9c2a5 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:59.561170 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59739a-ec45-415e-af97-acf0b4d9c2a5 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:59.562676 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 52/99] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:13:59.573340 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24c6ff-df07-484c-9299-243955cedc97 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:13:59.573632 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24c6ff-df07-484c-9299-243955cedc97 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:13:59.573916 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24c6ff-df07-484c-9299-243955cedc97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:13:59.582582 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 48/100] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:13:59.617818 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-9e24c6ff-df07-484c-9299-243955cedc97 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:13:59.619063 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-9e24c6ff-df07-484c-9299-243955cedc97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:13:59.709820 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 53/101] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:13:59.997497 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e536f3-e156-4f82-9147-93a5065afa13 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:00.100341 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:00.117229 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e536f3-e156-4f82-9147-93a5065afa13 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:00.153609 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5a3cf-f786-4f65-9bfe-3687498e0896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:00.154255 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5a3cf-f786-4f65-9bfe-3687498e0896 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:00.154536 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5a3cf-f786-4f65-9bfe-3687498e0896 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:00.154810 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5a3cf-f786-4f65-9bfe-3687498e0896 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:00.155940 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d5a3cf-f786-4f65-9bfe-3687498e0896 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:00.157413 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d5a3cf-f786-4f65-9bfe-3687498e0896 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:00.159232 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 49/102] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:13:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:00.165322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e24c6ff-df07-484c-9299-243955cedc97 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:00.174648 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:00.269766 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5a3cf-f786-4f65-9bfe-3687498e0896 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:00.307390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e24c6ff-df07-484c-9299-243955cedc97 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:00.390108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e208b-892d-4971-bb79-2ee7597c93df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:00.390755 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e208b-892d-4971-bb79-2ee7597c93df None admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:00.391038 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e208b-892d-4971-bb79-2ee7597c93df None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:00.391314 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e208b-892d-4971-bb79-2ee7597c93df None admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:00.392020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e208b-892d-4971-bb79-2ee7597c93df None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:00.393502 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e208b-892d-4971-bb79-2ee7597c93df None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:00.465919 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 54/103] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Aug 23 04:14:00 2022] POST /identity/v3/projects => generated 302 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:14:00.495273 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5a3cf-f786-4f65-9bfe-3687498e0896 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:00.548665 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9168bc8-b1e8-4a22-9063-82be24d476d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:00.549279 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9168bc8-b1e8-4a22-9063-82be24d476d2 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:00.549563 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9168bc8-b1e8-4a22-9063-82be24d476d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:00.549837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9168bc8-b1e8-4a22-9063-82be24d476d2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:00.569026 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9168bc8-b1e8-4a22-9063-82be24d476d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:00.574759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-c9168bc8-b1e8-4a22-9063-82be24d476d2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 04:14:00.574759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:00.574759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:00.574759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:00.574759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:00.574759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:00.574759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:00.574759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:00.574759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:00.574759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:00.574759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:00.574759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:00.574759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:00.574759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:00.574759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:00.574759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:00.575601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:00.575601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:00.575601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:00.575601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:00.575601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:00.575601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:00.575601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:00.575601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:00.575601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:00.575601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:00.575601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:00.575601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:00.575601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:00.575601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:00.575601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:00.575601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:00.576354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:00.576354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:00.576354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:00.576354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:00.576354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:00.576354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:00.576354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:00.576354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:00.576354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:00.576354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:00.576354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 04:14:00.576354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:00.576354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 50/104] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:14:00.583242 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e208b-892d-4971-bb79-2ee7597c93df None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:00.636741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4d82f-8d91-409a-8a8d-b8b8c0e8b8d0 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:00.637060 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4d82f-8d91-409a-8a8d-b8b8c0e8b8d0 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:00.637305 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4d82f-8d91-409a-8a8d-b8b8c0e8b8d0 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:00.638999 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 51/105] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:00.644596 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6095feb-453f-4e92-a682-afb9ffc5f3d4 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:00.644919 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6095feb-453f-4e92-a682-afb9ffc5f3d4 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:00.645164 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6095feb-453f-4e92-a682-afb9ffc5f3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:00.673841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-f6095feb-453f-4e92-a682-afb9ffc5f3d4 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:00.675330 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-f6095feb-453f-4e92-a682-afb9ffc5f3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:00.692352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:00.697707 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:00.767491 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e208b-892d-4971-bb79-2ee7597c93df None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:00.803184 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831239ca-3706-421f-b870-c74a85ca8ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:00.803750 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831239ca-3706-421f-b870-c74a85ca8ffe None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:00.804067 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831239ca-3706-421f-b870-c74a85ca8ffe None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:00.804347 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831239ca-3706-421f-b870-c74a85ca8ffe None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:00.805156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831239ca-3706-421f-b870-c74a85ca8ffe None admin] RBAC: Adding query filter params (name=reader) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:00.805613 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831239ca-3706-421f-b870-c74a85ca8ffe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:00.806442 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 52/106] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 04:14:00.807028 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831239ca-3706-421f-b870-c74a85ca8ffe None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:00.808122 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1570794-d73a-4031-b83e-a643931616e4 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:00.808666 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1570794-d73a-4031-b83e-a643931616e4 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:00.809142 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1570794-d73a-4031-b83e-a643931616e4 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:00.816465 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 53/108] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:00 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:00.816626 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 55/108] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Aug 23 04:14:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:00.821599 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4364023-daff-4999-be7c-6447fdc5d799 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:00.822101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4364023-daff-4999-be7c-6447fdc5d799 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:00.822483 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4364023-daff-4999-be7c-6447fdc5d799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:00.822657 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1570794-d73a-4031-b83e-a643931616e4 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:00.843133 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-a4364023-daff-4999-be7c-6447fdc5d799 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:00.844739 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-a4364023-daff-4999-be7c-6447fdc5d799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:00.941109 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 56/109] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:00.984095 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1570794-d73a-4031-b83e-a643931616e4 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:01.024100 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bb038-a8f1-4ffc-ae23-ced755d6eabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:01.024812 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bb038-a8f1-4ffc-ae23-ced755d6eabf None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:01.025098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bb038-a8f1-4ffc-ae23-ced755d6eabf None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:01.025378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bb038-a8f1-4ffc-ae23-ced755d6eabf None admin] PATH_INFO: `/v3/users/28038a4cdebf4eea929542b042210018` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:01.044082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:01.047432 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6bb038-a8f1-4ffc-ae23-ced755d6eabf None admin] RBAC: Authorizing `identity:get_user(user_id=28038a4cdebf4eea929542b042210018)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:01.049003 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6bb038-a8f1-4ffc-ae23-ced755d6eabf None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:01.050877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 54/110] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:14:00 2022] GET /identity/v3/users/28038a4cdebf4eea929542b042210018 => generated 289 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:01.058618 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4364023-daff-4999-be7c-6447fdc5d799 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:01.116298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:01.164208 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4364023-daff-4999-be7c-6447fdc5d799 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:01.168106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d5ed0-7c10-4985-bc6a-169c331b9973 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:01.168400 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d5ed0-7c10-4985-bc6a-169c331b9973 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:01.168680 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d5ed0-7c10-4985-bc6a-169c331b9973 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:01.170369 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 55/111] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:01.175113 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fd927-fa23-4a24-8441-2b3d6e915474 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:01.175400 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fd927-fa23-4a24-8441-2b3d6e915474 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:01.175678 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fd927-fa23-4a24-8441-2b3d6e915474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:01.205813 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-031fd927-fa23-4a24-8441-2b3d6e915474 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:01.207021 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-031fd927-fa23-4a24-8441-2b3d6e915474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:01.233976 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50964e63-a6d7-40b8-a10e-6c0aa8e633d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:01.234772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50964e63-a6d7-40b8-a10e-6c0aa8e633d1 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:01.235066 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50964e63-a6d7-40b8-a10e-6c0aa8e633d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:01.235335 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50964e63-a6d7-40b8-a10e-6c0aa8e633d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:01.310143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50964e63-a6d7-40b8-a10e-6c0aa8e633d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:01.311559 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50964e63-a6d7-40b8-a10e-6c0aa8e633d1 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:01.350135 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 56/112] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 04:14:01.351015 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031fd927-fa23-4a24-8441-2b3d6e915474 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:01.403320 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:01.424953 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 57/113] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:14:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:14:01.425838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50964e63-a6d7-40b8-a10e-6c0aa8e633d1 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:01.546884 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50964e63-a6d7-40b8-a10e-6c0aa8e633d1 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:01.550208 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031fd927-fa23-4a24-8441-2b3d6e915474 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:01.604176 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:01.612266 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8931ec7-0447-4999-b50c-2c531916e88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:01.613263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8931ec7-0447-4999-b50c-2c531916e88f None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:01.613814 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8931ec7-0447-4999-b50c-2c531916e88f None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:01.614226 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8931ec7-0447-4999-b50c-2c531916e88f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:01.626420 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8931ec7-0447-4999-b50c-2c531916e88f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:01.631391 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-d8931ec7-0447-4999-b50c-2c531916e88f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:01.631391 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:01.631391 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:01.631391 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:01.631391 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:01.631391 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:01.631391 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:01.631391 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:01.631391 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:01.631391 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:01.631391 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:01.631391 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:01.631391 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:01.631391 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:01.631391 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:01.631391 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:01.632254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:01.632254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:01.632254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:01.632254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:01.632254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:01.632254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:01.632254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:01.632254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:01.632254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:01.632254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:01.632254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:01.632254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:01.632254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:01.632254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:01.632254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:01.632254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:01.633034 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:01.633034 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:01.633034 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:01.633034 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:01.633034 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:01.633034 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:01.633034 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:01.633034 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:01.633034 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:01.633034 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:01.633034 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:01.633034 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:01.635982 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 57/114] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:01.637158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8931ec7-0447-4999-b50c-2c531916e88f None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:01.640328 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f1f37f-c64f-4c6c-9a83-3177f222b1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:01.640997 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1f37f-c64f-4c6c-9a83-3177f222b1b0 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:01.641274 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1f37f-c64f-4c6c-9a83-3177f222b1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:01.641696 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1f37f-c64f-4c6c-9a83-3177f222b1b0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:01.661101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f1f37f-c64f-4c6c-9a83-3177f222b1b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:01.666385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-00f1f37f-c64f-4c6c-9a83-3177f222b1b0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 04:14:01.666385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:01.666385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:01.666385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:01.666385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:01.666385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:01.666385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:01.666385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:01.666385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:01.666385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:01.666385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:01.666385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:01.666385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:01.666385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:01.666385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:01.666385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:01.667190 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:01.667190 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:01.667190 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:01.667190 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:01.667190 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:01.667190 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:01.667190 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:01.667190 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:01.667190 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:01.667190 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:01.667190 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:01.667190 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:01.667190 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:01.667190 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:01.667190 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:01.667190 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:01.667956 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:01.667956 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:01.667956 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:01.667956 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:01.667956 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:01.667956 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:01.667956 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:01.667956 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:01.667956 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:01.667956 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:01.667956 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 04:14:01.667956 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:01.670252 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 58/115] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Aug 23 04:14:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:01.675817 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f1f37f-c64f-4c6c-9a83-3177f222b1b0 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:01.690298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:01.762568 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8931ec7-0447-4999-b50c-2c531916e88f None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:01.799108 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbbed2-ce3f-4d29-9f01-bf2db76ead54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:01.800067 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbbed2-ce3f-4d29-9f01-bf2db76ead54 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:01.800535 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbbed2-ce3f-4d29-9f01-bf2db76ead54 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:01.800967 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbbed2-ce3f-4d29-9f01-bf2db76ead54 None admin] PATH_INFO: `/v3/projects/e3148f82c47e4702b119cf867f5dcdcc` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:01.802283 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bbbed2-ce3f-4d29-9f01-bf2db76ead54 None admin] RBAC: Authorizing `identity:get_project(project_id=e3148f82c47e4702b119cf867f5dcdcc)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:01.804094 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bbbed2-ce3f-4d29-9f01-bf2db76ead54 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:01.806217 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 58/116] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:01 2022] GET /identity/v3/projects/e3148f82c47e4702b119cf867f5dcdcc => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:01.807243 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbbed2-ce3f-4d29-9f01-bf2db76ead54 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:01.824588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f1f37f-c64f-4c6c-9a83-3177f222b1b0 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:01.868055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b3ba6-1fbe-4d54-ae38-d15511a233df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:01.868055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b3ba6-1fbe-4d54-ae38-d15511a233df None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:01.868055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b3ba6-1fbe-4d54-ae38-d15511a233df None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:01.868055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b3ba6-1fbe-4d54-ae38-d15511a233df None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:01.868565 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b3ba6-1fbe-4d54-ae38-d15511a233df None admin] RBAC: Adding query filter params (name=member) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:01.868565 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b3ba6-1fbe-4d54-ae38-d15511a233df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:01.869463 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b3ba6-1fbe-4d54-ae38-d15511a233df None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:01.877837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 59/117] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Aug 23 04:14:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:01.878566 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:01.878673 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b3ba6-1fbe-4d54-ae38-d15511a233df None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:01.926557 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbbed2-ce3f-4d29-9f01-bf2db76ead54 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:01.959264 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:01.963527 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741701b9-2a55-46cf-bc56-ce1941a329f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:01.964556 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741701b9-2a55-46cf-bc56-ce1941a329f3 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:01.964836 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741701b9-2a55-46cf-bc56-ce1941a329f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:01.965536 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741701b9-2a55-46cf-bc56-ce1941a329f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:01.976275 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741701b9-2a55-46cf-bc56-ce1941a329f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:01.981396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-741701b9-2a55-46cf-bc56-ce1941a329f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:14:01.981396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:01.981396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:01.981396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:01.981396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:01.981396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:01.981396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:01.981396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:01.981396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:01.981396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:01.981396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:01.981396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:01.981396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:01.981396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:01.981396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:01.981396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:01.982301 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:01.982301 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:01.982301 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:01.982301 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:01.982301 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:01.982301 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:01.982301 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:01.982301 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:01.982301 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:01.982301 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:01.982301 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:01.982301 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:01.982301 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:01.982301 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:01.982301 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:01.982301 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:01.983061 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:01.983061 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:01.983061 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:01.983061 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:01.983061 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:01.983061 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:01.983061 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:01.983061 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:01.983061 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:01.983061 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:01.983061 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:14:01.983061 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:01.983061 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 59/118] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:14:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:01.983649 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741701b9-2a55-46cf-bc56-ce1941a329f3 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:02.001697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b3ba6-1fbe-4d54-ae38-d15511a233df None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:02.039665 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62116b93-fb7d-4876-8d23-b6e74f9a550b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:02.040501 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62116b93-fb7d-4876-8d23-b6e74f9a550b None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:02.040809 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62116b93-fb7d-4876-8d23-b6e74f9a550b None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:02.041096 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62116b93-fb7d-4876-8d23-b6e74f9a550b None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:02.041733 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62116b93-fb7d-4876-8d23-b6e74f9a550b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:02.042045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62116b93-fb7d-4876-8d23-b6e74f9a550b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:02.043453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62116b93-fb7d-4876-8d23-b6e74f9a550b None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:02.051962 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 60/119] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:14:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:02.053307 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d795f9-98d8-4c1d-a1d2-97914075a734 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:02.053516 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d795f9-98d8-4c1d-a1d2-97914075a734 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:02.053772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d795f9-98d8-4c1d-a1d2-97914075a734 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:02.059356 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 61/120] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:02.059356 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d795f9-98d8-4c1d-a1d2-97914075a734 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:02.101480 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741701b9-2a55-46cf-bc56-ce1941a329f3 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:02.119891 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:02.138775 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870b306-9a53-42f4-abc0-0d34475f87e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:02.139411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870b306-9a53-42f4-abc0-0d34475f87e0 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:02.139667 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870b306-9a53-42f4-abc0-0d34475f87e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:02.139968 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870b306-9a53-42f4-abc0-0d34475f87e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:02.140711 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870b306-9a53-42f4-abc0-0d34475f87e0 None admin] RBAC: Adding query filter params (role.id=4be260163ba342e9b42232a49eec9fec, scope.project.id=e3148f82c47e4702b119cf867f5dcdcc, user.id=28038a4cdebf4eea929542b042210018) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:02.141019 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870b306-9a53-42f4-abc0-0d34475f87e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:02.142720 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870b306-9a53-42f4-abc0-0d34475f87e0 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:02.148341 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 60/121] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:14:01 2022] GET /identity/v3/role_assignments?user.id=28038a4cdebf4eea929542b042210018&scope.project.id=e3148f82c47e4702b119cf867f5dcdcc&role.id=4be260163ba342e9b42232a49eec9fec => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:02.149278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870b306-9a53-42f4-abc0-0d34475f87e0 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:02.178567 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d795f9-98d8-4c1d-a1d2-97914075a734 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:02.233153 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee5027-0574-4f0a-9316-5ff379308a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:02.233823 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee5027-0574-4f0a-9316-5ff379308a16 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:02.234343 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee5027-0574-4f0a-9316-5ff379308a16 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:02.236124 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee5027-0574-4f0a-9316-5ff379308a16 None admin] PATH_INFO: `/v3/users/628f03c8b9b44e27bacc6919029346d1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:02.238238 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee5027-0574-4f0a-9316-5ff379308a16 None admin] RBAC: Authorizing `identity:get_user(user_id=628f03c8b9b44e27bacc6919029346d1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:02.240734 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee5027-0574-4f0a-9316-5ff379308a16 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:02.249235 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 62/122] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:14:02 2022] GET /identity/v3/users/628f03c8b9b44e27bacc6919029346d1 => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:02.251271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee5027-0574-4f0a-9316-5ff379308a16 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:02.255975 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:02.291962 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870b306-9a53-42f4-abc0-0d34475f87e0 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:02.336588 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe10e47-973b-4f87-8e09-53664a68b74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:02.337243 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe10e47-973b-4f87-8e09-53664a68b74c None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:02.337553 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe10e47-973b-4f87-8e09-53664a68b74c None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:02.337813 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe10e47-973b-4f87-8e09-53664a68b74c None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:02.338500 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe10e47-973b-4f87-8e09-53664a68b74c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:02.338814 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe10e47-973b-4f87-8e09-53664a68b74c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:02.340704 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe10e47-973b-4f87-8e09-53664a68b74c None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:02.354491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 61/123] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:02.355971 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282248ac-c11b-4715-a8f0-21fdeb55ebc6 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:02.356273 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282248ac-c11b-4715-a8f0-21fdeb55ebc6 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:02.356554 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282248ac-c11b-4715-a8f0-21fdeb55ebc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:02.375602 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-282248ac-c11b-4715-a8f0-21fdeb55ebc6 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:02.376998 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-282248ac-c11b-4715-a8f0-21fdeb55ebc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:02.384230 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee5027-0574-4f0a-9316-5ff379308a16 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:02.421341 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:02.431528 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb0d10-5653-44d6-9eff-e33c0411a5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:02.432785 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb0d10-5653-44d6-9eff-e33c0411a5f3 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:02.433258 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb0d10-5653-44d6-9eff-e33c0411a5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:02.433954 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb0d10-5653-44d6-9eff-e33c0411a5f3 None admin] PATH_INFO: `/v3/users/628f03c8b9b44e27bacc6919029346d1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:02.435698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb0d10-5653-44d6-9eff-e33c0411a5f3 None admin] RBAC: Authorizing `identity:get_user(user_id=628f03c8b9b44e27bacc6919029346d1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:02.437430 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb0d10-5653-44d6-9eff-e33c0411a5f3 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:02.439996 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 63/124] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:14:02 2022] GET /identity/v3/users/628f03c8b9b44e27bacc6919029346d1 => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:02.440995 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb0d10-5653-44d6-9eff-e33c0411a5f3 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:02.465288 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 62/125] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:02.466600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282248ac-c11b-4715-a8f0-21fdeb55ebc6 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:02.541661 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:02.578015 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb0d10-5653-44d6-9eff-e33c0411a5f3 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:02.589721 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282248ac-c11b-4715-a8f0-21fdeb55ebc6 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:02.616079 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d2141-ccd6-4cb8-b56a-f9f7ff20201a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:02.616888 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d2141-ccd6-4cb8-b56a-f9f7ff20201a None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:02.617154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d2141-ccd6-4cb8-b56a-f9f7ff20201a None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:02.617481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d2141-ccd6-4cb8-b56a-f9f7ff20201a None admin] PATH_INFO: `/v3/projects/e3148f82c47e4702b119cf867f5dcdcc` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:02.618769 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d2141-ccd6-4cb8-b56a-f9f7ff20201a None admin] RBAC: Authorizing `identity:get_project(project_id=e3148f82c47e4702b119cf867f5dcdcc)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:02.620468 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d2141-ccd6-4cb8-b56a-f9f7ff20201a None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:02.622519 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 64/126] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:02 2022] GET /identity/v3/projects/e3148f82c47e4702b119cf867f5dcdcc => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:02.623295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d2141-ccd6-4cb8-b56a-f9f7ff20201a None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:02.634365 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c95e6-2891-43fc-9a12-e40cb0c4edb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:02.635322 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c95e6-2891-43fc-9a12-e40cb0c4edb1 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:02.635790 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c95e6-2891-43fc-9a12-e40cb0c4edb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:02.636291 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c95e6-2891-43fc-9a12-e40cb0c4edb1 None admin] PATH_INFO: `/v3/users/de65eb39f13f4a92a6a526fb5ab1f5c5` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:02.637604 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c95e6-2891-43fc-9a12-e40cb0c4edb1 None admin] RBAC: Authorizing `identity:get_user(user_id=de65eb39f13f4a92a6a526fb5ab1f5c5)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:02.639367 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c95e6-2891-43fc-9a12-e40cb0c4edb1 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:02.641204 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 63/127] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:14:02 2022] GET /identity/v3/users/de65eb39f13f4a92a6a526fb5ab1f5c5 => generated 247 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:02.642118 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c95e6-2891-43fc-9a12-e40cb0c4edb1 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:02.687451 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:02.750776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d2141-ccd6-4cb8-b56a-f9f7ff20201a None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:02.759521 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c95e6-2891-43fc-9a12-e40cb0c4edb1 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:02.788637 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ae40b-3e45-49d3-bcf0-07f53b9aa822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:02.789843 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ae40b-3e45-49d3-bcf0-07f53b9aa822 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:02.790394 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ae40b-3e45-49d3-bcf0-07f53b9aa822 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:02.790890 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ae40b-3e45-49d3-bcf0-07f53b9aa822 None admin] PATH_INFO: `/v3/projects/e3148f82c47e4702b119cf867f5dcdcc` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:02.792230 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ae40b-3e45-49d3-bcf0-07f53b9aa822 None admin] RBAC: Authorizing `identity:get_project(project_id=e3148f82c47e4702b119cf867f5dcdcc)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:02.793893 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ae40b-3e45-49d3-bcf0-07f53b9aa822 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:02.796142 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 65/128] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:02 2022] GET /identity/v3/projects/e3148f82c47e4702b119cf867f5dcdcc => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:02.797092 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ae40b-3e45-49d3-bcf0-07f53b9aa822 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:02.807178 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f495408-5767-4946-893e-27863e21ec11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:02.807892 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f495408-5767-4946-893e-27863e21ec11 None admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:02.808206 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f495408-5767-4946-893e-27863e21ec11 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:02.808482 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f495408-5767-4946-893e-27863e21ec11 None admin] PATH_INFO: `/v3/projects/e3148f82c47e4702b119cf867f5dcdcc/users/628f03c8b9b44e27bacc6919029346d1/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:02.808915 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:02.811073 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f495408-5767-4946-893e-27863e21ec11 None admin] RBAC: Authorizing `identity:create_grant(project_id=e3148f82c47e4702b119cf867f5dcdcc, user_id=628f03c8b9b44e27bacc6919029346d1, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:02.812700 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f495408-5767-4946-893e-27863e21ec11 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:02.830634 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 64/129] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Aug 23 04:14:02 2022] PUT /identity/v3/projects/e3148f82c47e4702b119cf867f5dcdcc/users/628f03c8b9b44e27bacc6919029346d1/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:14:02.836157 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f495408-5767-4946-893e-27863e21ec11 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:02.943210 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ae40b-3e45-49d3-bcf0-07f53b9aa822 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:02.994446 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f495408-5767-4946-893e-27863e21ec11 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:02.996054 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d014fcd5-6730-4e13-9e55-ad75123fcb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:02.997125 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:02.997399 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014fcd5-6730-4e13-9e55-ad75123fcb93 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:02.998487 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014fcd5-6730-4e13-9e55-ad75123fcb93 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:02.999951 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014fcd5-6730-4e13-9e55-ad75123fcb93 None admin] PATH_INFO: `/v3/projects/e3148f82c47e4702b119cf867f5dcdcc` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:03.001365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d014fcd5-6730-4e13-9e55-ad75123fcb93 None admin] RBAC: Authorizing `identity:get_project(project_id=e3148f82c47e4702b119cf867f5dcdcc)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:03.003052 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d014fcd5-6730-4e13-9e55-ad75123fcb93 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:03.004851 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 66/130] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:02 2022] GET /identity/v3/projects/e3148f82c47e4702b119cf867f5dcdcc => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:03.010424 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181a755-3bd6-4f8e-a188-ee3f71a1e29c None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:03.010424 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181a755-3bd6-4f8e-a188-ee3f71a1e29c None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:03.010424 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181a755-3bd6-4f8e-a188-ee3f71a1e29c None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:03.012707 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 67/131] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:03.013730 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181a755-3bd6-4f8e-a188-ee3f71a1e29c None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:03.048156 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ccaf1a-b055-488c-b72d-f72e95317edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:03.048839 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ccaf1a-b055-488c-b72d-f72e95317edf None admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:03.049126 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ccaf1a-b055-488c-b72d-f72e95317edf None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:03.049405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ccaf1a-b055-488c-b72d-f72e95317edf None admin] PATH_INFO: `/v3/projects/e3148f82c47e4702b119cf867f5dcdcc/users/628f03c8b9b44e27bacc6919029346d1/roles/4207a743edc04b0aba9197fe3554b2e3` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:03.052129 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ccaf1a-b055-488c-b72d-f72e95317edf None admin] RBAC: Authorizing `identity:create_grant(project_id=e3148f82c47e4702b119cf867f5dcdcc, user_id=628f03c8b9b44e27bacc6919029346d1, role_id=4207a743edc04b0aba9197fe3554b2e3)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:03.054279 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ccaf1a-b055-488c-b72d-f72e95317edf None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:03.073740 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 65/132] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Aug 23 04:14:02 2022] PUT /identity/v3/projects/e3148f82c47e4702b119cf867f5dcdcc/users/628f03c8b9b44e27bacc6919029346d1/roles/4207a743edc04b0aba9197fe3554b2e3 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:14:03.074813 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ccaf1a-b055-488c-b72d-f72e95317edf None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:03.118331 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:03.149893 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181a755-3bd6-4f8e-a188-ee3f71a1e29c None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:03.187866 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780bcb15-05c4-44dc-a3eb-e419c56325a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:03.188703 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bcb15-05c4-44dc-a3eb-e419c56325a6 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:03.189054 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bcb15-05c4-44dc-a3eb-e419c56325a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:03.189398 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bcb15-05c4-44dc-a3eb-e419c56325a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:03.194424 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ccaf1a-b055-488c-b72d-f72e95317edf None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:03.202025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780bcb15-05c4-44dc-a3eb-e419c56325a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:03.207099 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-780bcb15-05c4-44dc-a3eb-e419c56325a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:03.207099 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:03.207099 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:03.207099 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:03.207099 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:03.207099 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:03.207099 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:03.207099 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:03.207099 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:03.207099 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:03.207099 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:03.207099 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:03.207099 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:03.207099 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:03.207099 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:03.207099 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:03.207980 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:03.207980 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:03.207980 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:03.207980 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:03.207980 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:03.207980 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:03.207980 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:03.207980 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:03.207980 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:03.207980 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:03.207980 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:03.207980 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:03.207980 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:03.207980 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:03.207980 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:03.207980 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:03.208720 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:03.208720 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:03.208720 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:03.208720 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:03.208720 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:03.208720 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:03.208720 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:03.208720 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:03.208720 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:03.208720 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:03.208720 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:03.208720 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:03.208720 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 68/133] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:03.215035 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffd020-da63-4d02-b7c9-6c7b5208a88b None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:03.215441 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffd020-da63-4d02-b7c9-6c7b5208a88b None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:03.215838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffd020-da63-4d02-b7c9-6c7b5208a88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:03.236641 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-a9ffd020-da63-4d02-b7c9-6c7b5208a88b None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:03.237996 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-a9ffd020-da63-4d02-b7c9-6c7b5208a88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:03.238592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc764e-d98a-482b-8e44-19869b02ac14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:03.239483 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc764e-d98a-482b-8e44-19869b02ac14 None admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:03.239970 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc764e-d98a-482b-8e44-19869b02ac14 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:03.240451 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc764e-d98a-482b-8e44-19869b02ac14 None admin] PATH_INFO: `/v3/projects/e3148f82c47e4702b119cf867f5dcdcc/users/de65eb39f13f4a92a6a526fb5ab1f5c5/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:03.243072 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc764e-d98a-482b-8e44-19869b02ac14 None admin] RBAC: Authorizing `identity:create_grant(project_id=e3148f82c47e4702b119cf867f5dcdcc, user_id=de65eb39f13f4a92a6a526fb5ab1f5c5, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:03.244967 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc764e-d98a-482b-8e44-19869b02ac14 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:03.267831 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 66/134] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Aug 23 04:14:03 2022] PUT /identity/v3/projects/e3148f82c47e4702b119cf867f5dcdcc/users/de65eb39f13f4a92a6a526fb5ab1f5c5/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:14:03.269020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc764e-d98a-482b-8e44-19869b02ac14 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:03.327125 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:03.378138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 69/135] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:03.455828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc764e-d98a-482b-8e44-19869b02ac14 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:03.499049 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f07d687-55de-4120-86ae-11b92d6bce2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:03.499691 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07d687-55de-4120-86ae-11b92d6bce2c None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:03.500003 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07d687-55de-4120-86ae-11b92d6bce2c None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:03.500286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07d687-55de-4120-86ae-11b92d6bce2c None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:03.500983 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f07d687-55de-4120-86ae-11b92d6bce2c None admin] RBAC: Adding query filter params (name=member) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:03.501294 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f07d687-55de-4120-86ae-11b92d6bce2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:03.502704 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f07d687-55de-4120-86ae-11b92d6bce2c None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:03.512665 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 67/136] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Aug 23 04:14:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:03.518899 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffd020-da63-4d02-b7c9-6c7b5208a88b None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:03.586033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:03.625609 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffd020-da63-4d02-b7c9-6c7b5208a88b None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:03.660975 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46232e07-1c7f-4356-9990-e492677474b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:03.661635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46232e07-1c7f-4356-9990-e492677474b9 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:03.661916 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46232e07-1c7f-4356-9990-e492677474b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:03.662188 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46232e07-1c7f-4356-9990-e492677474b9 None admin] PATH_INFO: `/v3/users/628f03c8b9b44e27bacc6919029346d1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:03.663341 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46232e07-1c7f-4356-9990-e492677474b9 None admin] RBAC: Authorizing `identity:get_user(user_id=628f03c8b9b44e27bacc6919029346d1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:03.664954 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46232e07-1c7f-4356-9990-e492677474b9 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:03.666558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 70/137] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:14:03 2022] GET /identity/v3/users/628f03c8b9b44e27bacc6919029346d1 => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:03.672427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f07d687-55de-4120-86ae-11b92d6bce2c None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:03.762326 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:03.779430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f07d687-55de-4120-86ae-11b92d6bce2c None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:03.814529 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca8efb-a35f-4938-93bf-1713b2749da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:03.815171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca8efb-a35f-4938-93bf-1713b2749da2 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:03.815455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca8efb-a35f-4938-93bf-1713b2749da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:03.815732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca8efb-a35f-4938-93bf-1713b2749da2 None admin] PATH_INFO: `/v3/projects/67f78794d6ab4968bc00d61a6015220c` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:03.816888 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca8efb-a35f-4938-93bf-1713b2749da2 None admin] RBAC: Authorizing `identity:get_project(project_id=67f78794d6ab4968bc00d61a6015220c)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:03.818358 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca8efb-a35f-4938-93bf-1713b2749da2 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:03.820016 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 68/138] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:03 2022] GET /identity/v3/projects/67f78794d6ab4968bc00d61a6015220c => generated 312 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:03.832056 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46232e07-1c7f-4356-9990-e492677474b9 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:03.910257 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca8efb-a35f-4938-93bf-1713b2749da2 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:04.012772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46232e07-1c7f-4356-9990-e492677474b9 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:04.044133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca8efb-a35f-4938-93bf-1713b2749da2 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:04.080563 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09365cb1-573b-429e-9834-a057574074f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:04.082601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09365cb1-573b-429e-9834-a057574074f6 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:04.082601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09365cb1-573b-429e-9834-a057574074f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:04.082601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09365cb1-573b-429e-9834-a057574074f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:04.083519 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09365cb1-573b-429e-9834-a057574074f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:04.085190 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09365cb1-573b-429e-9834-a057574074f6 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:04.087598 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 69/139] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:14:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:14:04.094162 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09365cb1-573b-429e-9834-a057574074f6 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:04.097624 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:04.108051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6322e-405d-41a0-b5a6-f8527dd6c06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:04.109046 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6322e-405d-41a0-b5a6-f8527dd6c06a None admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:04.109538 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6322e-405d-41a0-b5a6-f8527dd6c06a None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:04.109993 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6322e-405d-41a0-b5a6-f8527dd6c06a None admin] PATH_INFO: `/v3/projects/67f78794d6ab4968bc00d61a6015220c/users/628f03c8b9b44e27bacc6919029346d1/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:04.112231 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc6322e-405d-41a0-b5a6-f8527dd6c06a None admin] RBAC: Authorizing `identity:create_grant(project_id=67f78794d6ab4968bc00d61a6015220c, user_id=628f03c8b9b44e27bacc6919029346d1, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:04.121328 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc6322e-405d-41a0-b5a6-f8527dd6c06a None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:04.128361 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:04.154854 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 71/140] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Aug 23 04:14:03 2022] PUT /identity/v3/projects/67f78794d6ab4968bc00d61a6015220c/users/628f03c8b9b44e27bacc6919029346d1/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:14:04.206530 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09365cb1-573b-429e-9834-a057574074f6 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:04.245415 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e2dec-57c9-44e1-95dd-1d84021ce5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:04.246193 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e2dec-57c9-44e1-95dd-1d84021ce5ef None admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:04.246482 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e2dec-57c9-44e1-95dd-1d84021ce5ef None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:04.246833 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e2dec-57c9-44e1-95dd-1d84021ce5ef None admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:04.247507 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7e2dec-57c9-44e1-95dd-1d84021ce5ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:04.249072 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7e2dec-57c9-44e1-95dd-1d84021ce5ef None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:04.289196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 70/141] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Aug 23 04:14:04 2022] POST /identity/v3/users => generated 283 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:14:04.547313 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94ca70-0df8-4cb9-89cc-9f01a5c5b63c None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:04.547950 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94ca70-0df8-4cb9-89cc-9f01a5c5b63c None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:04.548440 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94ca70-0df8-4cb9-89cc-9f01a5c5b63c None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:04.550343 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 72/142] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:04.557475 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5eb4d6-eb5a-47ad-889e-2f69e6166eaa None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:04.558082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5eb4d6-eb5a-47ad-889e-2f69e6166eaa None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:04.558569 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5eb4d6-eb5a-47ad-889e-2f69e6166eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:04.577674 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-2c5eb4d6-eb5a-47ad-889e-2f69e6166eaa None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:04.579246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-2c5eb4d6-eb5a-47ad-889e-2f69e6166eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:04.650819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 71/143] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:04.998199 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da94ca70-0df8-4cb9-89cc-9f01a5c5b63c None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:05.144610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da94ca70-0df8-4cb9-89cc-9f01a5c5b63c None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:05.176041 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc641c0-dbf0-47f1-86cd-842261fa922a None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:05.177033 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc641c0-dbf0-47f1-86cd-842261fa922a None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:05.177510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc641c0-dbf0-47f1-86cd-842261fa922a None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:05.179211 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 72/144] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:05.189223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3346a1a0-b2a3-41e2-b87b-ec351e6315ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:05.189857 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346a1a0-b2a3-41e2-b87b-ec351e6315ee None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:05.190129 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346a1a0-b2a3-41e2-b87b-ec351e6315ee None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:05.190396 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346a1a0-b2a3-41e2-b87b-ec351e6315ee None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:05.195783 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65a08c-877c-45f1-aba9-aaeda5c73da4 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:05.196579 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65a08c-877c-45f1-aba9-aaeda5c73da4 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:05.197089 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65a08c-877c-45f1-aba9-aaeda5c73da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:05.208241 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3346a1a0-b2a3-41e2-b87b-ec351e6315ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:05.212284 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:05.212982 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-3346a1a0-b2a3-41e2-b87b-ec351e6315ee None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 04:14:05.212982 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:05.212982 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:05.212982 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:05.212982 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:05.212982 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:05.212982 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:05.212982 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:05.212982 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:05.212982 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:05.212982 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:05.212982 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:05.212982 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:05.212982 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:05.212982 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:05.212982 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:05.213803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:05.213803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:05.213803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:05.213803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:05.213803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:05.213803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:05.213803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:05.213803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:05.213803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:05.213803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:05.213803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:05.213803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:05.213803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:05.213803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:05.213803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:05.213803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:05.214903 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:05.214903 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:05.214903 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:05.214903 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:05.214903 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:05.214903 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:05.214903 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:05.214903 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:05.214903 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:05.214903 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:05.214903 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 04:14:05.214903 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:05.214903 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 73/145] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:05.222166 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3346a1a0-b2a3-41e2-b87b-ec351e6315ee None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:05.225000 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-ce65a08c-877c-45f1-aba9-aaeda5c73da4 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:05.226320 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-ce65a08c-877c-45f1-aba9-aaeda5c73da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:05.303274 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 73/146] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 04:14:05.310284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:05.371048 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3346a1a0-b2a3-41e2-b87b-ec351e6315ee None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:05.413609 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec31d6-0a6a-48e1-b2ca-3fb1cd311917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:05.414526 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec31d6-0a6a-48e1-b2ca-3fb1cd311917 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:05.414971 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec31d6-0a6a-48e1-b2ca-3fb1cd311917 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:05.415376 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec31d6-0a6a-48e1-b2ca-3fb1cd311917 None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:05.416207 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec31d6-0a6a-48e1-b2ca-3fb1cd311917 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:05.416642 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec31d6-0a6a-48e1-b2ca-3fb1cd311917 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:05.418162 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec31d6-0a6a-48e1-b2ca-3fb1cd311917 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:05.427868 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 74/147] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Aug 23 04:14:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:14:05.438153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65a08c-877c-45f1-aba9-aaeda5c73da4 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:05.567175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65a08c-877c-45f1-aba9-aaeda5c73da4 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:05.605653 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9fbf47-ed00-4bad-80da-076391afa136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:05.606296 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9fbf47-ed00-4bad-80da-076391afa136 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:05.606572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9fbf47-ed00-4bad-80da-076391afa136 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:05.606838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9fbf47-ed00-4bad-80da-076391afa136 None admin] PATH_INFO: `/v3/users/28038a4cdebf4eea929542b042210018` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:05.612604 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9fbf47-ed00-4bad-80da-076391afa136 None admin] RBAC: Authorizing `identity:get_user(user_id=28038a4cdebf4eea929542b042210018)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:05.614067 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9fbf47-ed00-4bad-80da-076391afa136 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:05.615687 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 74/148] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:14:05 2022] GET /identity/v3/users/28038a4cdebf4eea929542b042210018 => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:05.622191 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec31d6-0a6a-48e1-b2ca-3fb1cd311917 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:05.834329 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9fbf47-ed00-4bad-80da-076391afa136 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:05.834491 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec31d6-0a6a-48e1-b2ca-3fb1cd311917 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:05.841553 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:05.857880 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:05.897315 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d91ca0-4a3e-456b-ad66-3ba67c1e2110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:05.898262 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d91ca0-4a3e-456b-ad66-3ba67c1e2110 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:05.898626 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d91ca0-4a3e-456b-ad66-3ba67c1e2110 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:05.898903 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d91ca0-4a3e-456b-ad66-3ba67c1e2110 None admin] PATH_INFO: `/v3/projects/e3148f82c47e4702b119cf867f5dcdcc` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:05.901835 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d91ca0-4a3e-456b-ad66-3ba67c1e2110 None admin] RBAC: Authorizing `identity:get_project(project_id=e3148f82c47e4702b119cf867f5dcdcc)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:05.903317 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d91ca0-4a3e-456b-ad66-3ba67c1e2110 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:05.905169 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 75/149] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:05 2022] GET /identity/v3/projects/e3148f82c47e4702b119cf867f5dcdcc => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:05.913526 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d91ca0-4a3e-456b-ad66-3ba67c1e2110 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:06.034846 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9fbf47-ed00-4bad-80da-076391afa136 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:06.094482 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d91ca0-4a3e-456b-ad66-3ba67c1e2110 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:06.108357 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d824d54-ca12-4fa6-9754-973e734e66dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:06.109015 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d824d54-ca12-4fa6-9754-973e734e66dc None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:06.109291 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d824d54-ca12-4fa6-9754-973e734e66dc None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:06.109564 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d824d54-ca12-4fa6-9754-973e734e66dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:06.129842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d824d54-ca12-4fa6-9754-973e734e66dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:06.135098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-6d824d54-ca12-4fa6-9754-973e734e66dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:06.135098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:06.135098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:06.135098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:06.135098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:06.135098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:06.135098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:06.135098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:06.135098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:06.135098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:06.135098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:06.135098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:06.135098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:06.135098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:06.135098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:06.135098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:06.135959 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:06.135959 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:06.135959 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:06.135959 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:06.135959 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:06.135959 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:06.135959 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:06.135959 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:06.135959 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:06.135959 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:06.135959 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:06.135959 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:06.135959 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:06.135959 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:06.135959 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:06.135959 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:06.136714 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:06.136714 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:06.136714 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:06.136714 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:06.136714 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:06.136714 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:06.136714 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:06.136714 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:06.136714 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:06.136714 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:06.136714 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:06.136714 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:06.136714 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 75/150] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:06.145406 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d824d54-ca12-4fa6-9754-973e734e66dc None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:06.160213 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c220d3-25bc-46f9-86d2-9ec35d9dcc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:06.160973 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c220d3-25bc-46f9-86d2-9ec35d9dcc68 None admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:06.161247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c220d3-25bc-46f9-86d2-9ec35d9dcc68 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:06.161521 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c220d3-25bc-46f9-86d2-9ec35d9dcc68 None admin] PATH_INFO: `/v3/projects/e3148f82c47e4702b119cf867f5dcdcc/users/28038a4cdebf4eea929542b042210018/roles/4be260163ba342e9b42232a49eec9fec` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:06.163543 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c220d3-25bc-46f9-86d2-9ec35d9dcc68 None admin] RBAC: Authorizing `identity:create_grant(project_id=e3148f82c47e4702b119cf867f5dcdcc, user_id=28038a4cdebf4eea929542b042210018, role_id=4be260163ba342e9b42232a49eec9fec)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:06.165146 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c220d3-25bc-46f9-86d2-9ec35d9dcc68 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:06.199406 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 76/151] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Aug 23 04:14:05 2022] PUT /identity/v3/projects/e3148f82c47e4702b119cf867f5dcdcc/users/28038a4cdebf4eea929542b042210018/roles/4be260163ba342e9b42232a49eec9fec => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:14:06.249017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:06.285944 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d824d54-ca12-4fa6-9754-973e734e66dc None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:06.323974 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c41f1-ce66-4438-9bb6-47910c48b784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:06.324751 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c41f1-ce66-4438-9bb6-47910c48b784 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:06.325098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c41f1-ce66-4438-9bb6-47910c48b784 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:06.325436 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c41f1-ce66-4438-9bb6-47910c48b784 None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:06.326193 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930c41f1-ce66-4438-9bb6-47910c48b784 None admin] RBAC: Adding query filter params (name=member) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:06.326620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930c41f1-ce66-4438-9bb6-47910c48b784 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:06.328177 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930c41f1-ce66-4438-9bb6-47910c48b784 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:06.338321 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 76/152] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Aug 23 04:14:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:06.344659 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c220d3-25bc-46f9-86d2-9ec35d9dcc68 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:06.394998 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:06.478100 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c220d3-25bc-46f9-86d2-9ec35d9dcc68 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:06.537846 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a0761e-986d-46b9-be56-83d1dd241f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:06.538502 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0761e-986d-46b9-be56-83d1dd241f96 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:06.538780 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0761e-986d-46b9-be56-83d1dd241f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:06.539066 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0761e-986d-46b9-be56-83d1dd241f96 None admin] PATH_INFO: `/v3/users/628f03c8b9b44e27bacc6919029346d1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:06.544690 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a0761e-986d-46b9-be56-83d1dd241f96 None admin] RBAC: Authorizing `identity:get_user(user_id=628f03c8b9b44e27bacc6919029346d1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:06.546211 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a0761e-986d-46b9-be56-83d1dd241f96 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:06.547889 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 77/153] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:14:06 2022] GET /identity/v3/users/628f03c8b9b44e27bacc6919029346d1 => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:06.553917 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c41f1-ce66-4438-9bb6-47910c48b784 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:06.596612 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41930595-46aa-4299-ad30-9b493490494c None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:06.596899 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41930595-46aa-4299-ad30-9b493490494c None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:06.597085 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41930595-46aa-4299-ad30-9b493490494c None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:06.598449 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 78/154] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:06.606889 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27561eef-48d0-42c2-b3d5-754b0657d744 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:06.607174 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27561eef-48d0-42c2-b3d5-754b0657d744 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:06.607442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27561eef-48d0-42c2-b3d5-754b0657d744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:06.644065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-27561eef-48d0-42c2-b3d5-754b0657d744 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:06.645299 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-27561eef-48d0-42c2-b3d5-754b0657d744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:06.701361 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c41f1-ce66-4438-9bb6-47910c48b784 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:06.758907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb33c2-29ad-429b-b19e-ca34b46770df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:06.759607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb33c2-29ad-429b-b19e-ca34b46770df None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:06.764721 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb33c2-29ad-429b-b19e-ca34b46770df None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:06.765012 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb33c2-29ad-429b-b19e-ca34b46770df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:06.771137 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:06.806157 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 79/155] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 04:14:06.860047 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bb33c2-29ad-429b-b19e-ca34b46770df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:06.861447 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bb33c2-29ad-429b-b19e-ca34b46770df None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:06.897535 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88d179-9ca8-4444-98b1-00abe1c27bb2 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:06.897855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88d179-9ca8-4444-98b1-00abe1c27bb2 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:06.898098 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88d179-9ca8-4444-98b1-00abe1c27bb2 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:06.904568 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 80/156] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:06.913287 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365953db-57fd-456a-9258-126022a0bec1 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:06.913571 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365953db-57fd-456a-9258-126022a0bec1 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:06.913845 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365953db-57fd-456a-9258-126022a0bec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:06.961064 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-365953db-57fd-456a-9258-126022a0bec1 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:06.965828 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-365953db-57fd-456a-9258-126022a0bec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:06.997201 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 77/157] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:14:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:14:07.002600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb33c2-29ad-429b-b19e-ca34b46770df None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:07.039968 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:07.056287 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:07.117618 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 81/158] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:07.156039 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb33c2-29ad-429b-b19e-ca34b46770df None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:07.205674 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c6ea9-ba09-4f66-91ac-a867823dd74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:07.206352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c6ea9-ba09-4f66-91ac-a867823dd74e None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:07.206632 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c6ea9-ba09-4f66-91ac-a867823dd74e None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:07.206905 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c6ea9-ba09-4f66-91ac-a867823dd74e None admin] PATH_INFO: `/v3/projects/e3148f82c47e4702b119cf867f5dcdcc` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:07.208099 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c6ea9-ba09-4f66-91ac-a867823dd74e None admin] RBAC: Authorizing `identity:get_project(project_id=e3148f82c47e4702b119cf867f5dcdcc)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:07.209584 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c6ea9-ba09-4f66-91ac-a867823dd74e None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:07.216393 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 78/159] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:07 2022] GET /identity/v3/projects/e3148f82c47e4702b119cf867f5dcdcc => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:07.222242 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365953db-57fd-456a-9258-126022a0bec1 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:07.344730 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7ab16-c8dc-4d8f-afa8-e6c5e03d568c None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:07.345277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7ab16-c8dc-4d8f-afa8-e6c5e03d568c None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:07.345277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7ab16-c8dc-4d8f-afa8-e6c5e03d568c None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:07.346607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 79/160] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:14:07.364749 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365953db-57fd-456a-9258-126022a0bec1 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:07.384933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7ab16-c8dc-4d8f-afa8-e6c5e03d568c None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:07.414862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2dbd1b-379a-483d-a19a-4deb84991e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:07.415483 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2dbd1b-379a-483d-a19a-4deb84991e39 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:07.415758 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2dbd1b-379a-483d-a19a-4deb84991e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:07.416681 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2dbd1b-379a-483d-a19a-4deb84991e39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:07.417429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2dbd1b-379a-483d-a19a-4deb84991e39 None admin] RBAC: Adding query filter params (role.id=aa03504b4fb642d288d07f4335e291e1, scope.project.id=e3148f82c47e4702b119cf867f5dcdcc, user.id=628f03c8b9b44e27bacc6919029346d1) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:07.417732 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2dbd1b-379a-483d-a19a-4deb84991e39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:07.419191 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2dbd1b-379a-483d-a19a-4deb84991e39 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:07.437115 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 82/161] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:14:07 2022] GET /identity/v3/role_assignments?user.id=628f03c8b9b44e27bacc6919029346d1&scope.project.id=e3148f82c47e4702b119cf867f5dcdcc&role.id=aa03504b4fb642d288d07f4335e291e1 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:07.438384 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed053d-2159-41d5-8eec-519eeb59c8f0 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:07.438662 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed053d-2159-41d5-8eec-519eeb59c8f0 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:07.438932 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed053d-2159-41d5-8eec-519eeb59c8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:07.493379 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-ceed053d-2159-41d5-8eec-519eeb59c8f0 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:07.494598 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-ceed053d-2159-41d5-8eec-519eeb59c8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:07.538021 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:07.595480 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:07.616704 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7ab16-c8dc-4d8f-afa8-e6c5e03d568c None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:07.632048 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 83/162] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:07.683116 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42faf4c0-3811-4473-ac56-5b220896a1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:07.683847 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42faf4c0-3811-4473-ac56-5b220896a1c6 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:07.684165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42faf4c0-3811-4473-ac56-5b220896a1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:07.684440 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42faf4c0-3811-4473-ac56-5b220896a1c6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:07.701977 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42faf4c0-3811-4473-ac56-5b220896a1c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:07.706892 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-42faf4c0-3811-4473-ac56-5b220896a1c6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 04:14:07.706892 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:07.706892 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:07.706892 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:07.706892 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:07.706892 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:07.706892 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:07.706892 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:07.706892 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:07.706892 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:07.706892 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:07.706892 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:07.706892 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:07.706892 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:07.706892 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:07.706892 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:07.707744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:07.707744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:07.707744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:07.707744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:07.707744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:07.707744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:07.707744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:07.707744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:07.707744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:07.707744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:07.707744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:07.707744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:07.707744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:07.707744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:07.707744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:07.707744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:07.708566 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:07.708566 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:07.708566 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:07.708566 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:07.708566 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:07.708566 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:07.708566 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:07.708566 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:07.708566 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:07.708566 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:07.708566 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 04:14:07.708566 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:07.708566 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 80/163] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Aug 23 04:14:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:07.714531 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed053d-2159-41d5-8eec-519eeb59c8f0 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:07.745638 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42faf4c0-3811-4473-ac56-5b220896a1c6 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:07.927432 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed053d-2159-41d5-8eec-519eeb59c8f0 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:07.967612 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42faf4c0-3811-4473-ac56-5b220896a1c6 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:08.000216 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4511834-1a51-4dee-a1f8-b85a25dfb56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:08.000851 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4511834-1a51-4dee-a1f8-b85a25dfb56f None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:08.001132 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4511834-1a51-4dee-a1f8-b85a25dfb56f None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:08.001402 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4511834-1a51-4dee-a1f8-b85a25dfb56f None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:08.002099 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4511834-1a51-4dee-a1f8-b85a25dfb56f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:08.002411 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4511834-1a51-4dee-a1f8-b85a25dfb56f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:08.003807 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4511834-1a51-4dee-a1f8-b85a25dfb56f None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:08.025064 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 84/164] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:14:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:08.031783 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4511834-1a51-4dee-a1f8-b85a25dfb56f None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:08.049057 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f95e3-f8cc-4c67-9b6b-a1fbf386c2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:08.049702 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f95e3-f8cc-4c67-9b6b-a1fbf386c2fe None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:08.050043 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f95e3-f8cc-4c67-9b6b-a1fbf386c2fe None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:08.050317 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f95e3-f8cc-4c67-9b6b-a1fbf386c2fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:08.075748 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068f95e3-f8cc-4c67-9b6b-a1fbf386c2fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:08.083750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-068f95e3-f8cc-4c67-9b6b-a1fbf386c2fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:14:08.083750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:08.083750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:08.083750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:08.083750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:08.083750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:08.083750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:08.083750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:08.083750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:08.083750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:08.083750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:08.083750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:08.083750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:08.083750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:08.083750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:08.083750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:08.084639 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:08.084639 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:08.084639 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:08.084639 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:08.084639 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:08.084639 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:08.084639 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:08.084639 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:08.084639 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:08.084639 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:08.084639 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:08.084639 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:08.084639 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:08.084639 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:08.084639 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:08.084639 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:08.085379 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:08.085379 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:08.085379 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:08.085379 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:08.085379 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:08.085379 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:08.085379 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:08.085379 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:08.085379 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:08.085379 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:08.085379 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:14:08.085379 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:08.085379 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 81/165] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:14:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:14:08.092099 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f95e3-f8cc-4c67-9b6b-a1fbf386c2fe None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:08.095452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:08.128290 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:08.219441 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4511834-1a51-4dee-a1f8-b85a25dfb56f None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:08.256619 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f95e3-f8cc-4c67-9b6b-a1fbf386c2fe None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:08.282801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249cd146-f8a0-485e-a346-5787465a1652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:08.283465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cd146-f8a0-485e-a346-5787465a1652 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:08.283747 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cd146-f8a0-485e-a346-5787465a1652 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:08.284048 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cd146-f8a0-485e-a346-5787465a1652 None admin] PATH_INFO: `/v3/users/628f03c8b9b44e27bacc6919029346d1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:08.287192 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249cd146-f8a0-485e-a346-5787465a1652 None admin] RBAC: Authorizing `identity:get_user(user_id=628f03c8b9b44e27bacc6919029346d1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:08.288717 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249cd146-f8a0-485e-a346-5787465a1652 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:08.290662 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 85/166] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:14:08 2022] GET /identity/v3/users/628f03c8b9b44e27bacc6919029346d1 => generated 275 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:08.295589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249cd146-f8a0-485e-a346-5787465a1652 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:08.339971 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b78abc3-48ec-47cb-bb60-f28030caa18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:08.340611 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78abc3-48ec-47cb-bb60-f28030caa18a None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:08.341268 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78abc3-48ec-47cb-bb60-f28030caa18a None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:08.341687 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78abc3-48ec-47cb-bb60-f28030caa18a None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:08.342505 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b78abc3-48ec-47cb-bb60-f28030caa18a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:08.342957 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b78abc3-48ec-47cb-bb60-f28030caa18a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:08.344467 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b78abc3-48ec-47cb-bb60-f28030caa18a None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:08.366822 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 82/167] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:08.368317 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b78abc3-48ec-47cb-bb60-f28030caa18a None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:08.461851 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:08.469769 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249cd146-f8a0-485e-a346-5787465a1652 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:08.507152 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b4e7d1-1a31-4678-9eb2-022cabc94e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:08.507825 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4e7d1-1a31-4678-9eb2-022cabc94e13 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:08.508149 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4e7d1-1a31-4678-9eb2-022cabc94e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:08.508628 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4e7d1-1a31-4678-9eb2-022cabc94e13 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:08.519134 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b4e7d1-1a31-4678-9eb2-022cabc94e13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:08.524395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-68b4e7d1-1a31-4678-9eb2-022cabc94e13 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:08.524395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:08.524395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:08.524395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:08.524395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:08.524395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:08.524395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:08.524395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:08.524395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:08.524395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:08.524395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:08.524395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:08.524395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:08.524395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:08.524395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:08.524395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:08.525249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:08.525249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:08.525249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:08.525249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:08.525249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:08.525249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:08.525249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:08.525249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:08.525249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:08.525249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:08.525249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:08.525249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:08.525249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:08.525249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:08.525249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:08.525249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:08.526084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:08.526084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:08.526084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:08.526084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:08.526084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:08.526084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:08.526084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:08.526084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:08.526084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:08.526084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:08.526084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:08.526084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:08.526084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 86/168] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:08.526718 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b4e7d1-1a31-4678-9eb2-022cabc94e13 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:08.564314 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:08.637002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b78abc3-48ec-47cb-bb60-f28030caa18a None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:08.644866 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b4e7d1-1a31-4678-9eb2-022cabc94e13 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:08.679698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea958772-209c-4a71-bec8-cfeb14fe7507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:08.680769 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea958772-209c-4a71-bec8-cfeb14fe7507 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:08.680769 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea958772-209c-4a71-bec8-cfeb14fe7507 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:08.680960 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea958772-209c-4a71-bec8-cfeb14fe7507 None admin] PATH_INFO: `/v3/users/de65eb39f13f4a92a6a526fb5ab1f5c5` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:08.682042 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea958772-209c-4a71-bec8-cfeb14fe7507 None admin] RBAC: Authorizing `identity:get_user(user_id=de65eb39f13f4a92a6a526fb5ab1f5c5)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:08.683496 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea958772-209c-4a71-bec8-cfeb14fe7507 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:08.685434 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 87/169] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:14:08 2022] GET /identity/v3/users/de65eb39f13f4a92a6a526fb5ab1f5c5 => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:08.686284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea958772-209c-4a71-bec8-cfeb14fe7507 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:08.697249 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f95b93-c04e-4fb7-b683-fde265f0ea1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:08.697898 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f95b93-c04e-4fb7-b683-fde265f0ea1b None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:08.698181 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f95b93-c04e-4fb7-b683-fde265f0ea1b None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:08.698457 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f95b93-c04e-4fb7-b683-fde265f0ea1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:08.713675 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:08.795315 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea958772-209c-4a71-bec8-cfeb14fe7507 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:08.824616 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f95b93-c04e-4fb7-b683-fde265f0ea1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:08.826123 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f95b93-c04e-4fb7-b683-fde265f0ea1b None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:08.831400 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0bdd34-68b8-4907-b1a1-ff19a8d5832c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:08.832220 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0bdd34-68b8-4907-b1a1-ff19a8d5832c None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:08.833518 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0bdd34-68b8-4907-b1a1-ff19a8d5832c None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:08.833518 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0bdd34-68b8-4907-b1a1-ff19a8d5832c None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:08.833680 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0bdd34-68b8-4907-b1a1-ff19a8d5832c None admin] RBAC: Adding query filter params (name=member) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:08.833900 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0bdd34-68b8-4907-b1a1-ff19a8d5832c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:08.835346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0bdd34-68b8-4907-b1a1-ff19a8d5832c None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:08.845938 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 88/170] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Aug 23 04:14:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:08.849015 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0bdd34-68b8-4907-b1a1-ff19a8d5832c None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:08.939870 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:08.958118 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0bdd34-68b8-4907-b1a1-ff19a8d5832c None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:08.964475 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:09.002907 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d8b20-3520-43da-bc3e-ab11fda78cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:09.003543 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d8b20-3520-43da-bc3e-ab11fda78cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:09.003820 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d8b20-3520-43da-bc3e-ab11fda78cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:09.004122 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d8b20-3520-43da-bc3e-ab11fda78cc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:09.009193 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 83/171] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:14:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:14:09.013590 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f95b93-c04e-4fb7-b683-fde265f0ea1b None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:09.133815 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d8b20-3520-43da-bc3e-ab11fda78cc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:09.141136 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d8b20-3520-43da-bc3e-ab11fda78cc0 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:09.181767 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f95b93-c04e-4fb7-b683-fde265f0ea1b None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:09.228491 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:09.247577 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 89/172] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:14:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:14:09.249197 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42365b48-5e8e-4458-b74f-9292663892b1 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:09.249692 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42365b48-5e8e-4458-b74f-9292663892b1 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:09.250174 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42365b48-5e8e-4458-b74f-9292663892b1 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:09.251598 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 90/173] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:09.252506 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42365b48-5e8e-4458-b74f-9292663892b1 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:09.269478 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f8a19-858e-4888-a7b6-90820c0df2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:09.269924 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f8a19-858e-4888-a7b6-90820c0df2c9 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:09.270205 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f8a19-858e-4888-a7b6-90820c0df2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:09.270473 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f8a19-858e-4888-a7b6-90820c0df2c9 None admin] PATH_INFO: `/v3/users/628f03c8b9b44e27bacc6919029346d1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:09.276654 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f8a19-858e-4888-a7b6-90820c0df2c9 None admin] RBAC: Authorizing `identity:get_user(user_id=628f03c8b9b44e27bacc6919029346d1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:09.278129 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f8a19-858e-4888-a7b6-90820c0df2c9 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:09.283938 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 84/174] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:14:09 2022] GET /identity/v3/users/628f03c8b9b44e27bacc6919029346d1 => generated 275 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:09.283938 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b763c9-93f9-4899-8bff-5525282fda76 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:09.283938 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b763c9-93f9-4899-8bff-5525282fda76 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:09.283938 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b763c9-93f9-4899-8bff-5525282fda76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:09.324618 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-d2b763c9-93f9-4899-8bff-5525282fda76 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:09.325827 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-d2b763c9-93f9-4899-8bff-5525282fda76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:09.373120 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42365b48-5e8e-4458-b74f-9292663892b1 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:09.381081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:09.405054 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 85/175] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:09.406553 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b763c9-93f9-4899-8bff-5525282fda76 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:09.413536 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fac299-d1db-42e4-a57d-227997f92d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:09.414458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fac299-d1db-42e4-a57d-227997f92d8b None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:09.414954 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fac299-d1db-42e4-a57d-227997f92d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:09.415464 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fac299-d1db-42e4-a57d-227997f92d8b None admin] PATH_INFO: `/v3/projects/e3148f82c47e4702b119cf867f5dcdcc` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:09.416865 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fac299-d1db-42e4-a57d-227997f92d8b None admin] RBAC: Authorizing `identity:get_project(project_id=e3148f82c47e4702b119cf867f5dcdcc)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:09.418621 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fac299-d1db-42e4-a57d-227997f92d8b None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:09.420571 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 91/176] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:09 2022] GET /identity/v3/projects/e3148f82c47e4702b119cf867f5dcdcc => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:09.421560 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fac299-d1db-42e4-a57d-227997f92d8b None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:09.504737 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:09.544938 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b763c9-93f9-4899-8bff-5525282fda76 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:09.592655 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fac299-d1db-42e4-a57d-227997f92d8b None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:09.629377 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c704a-15a0-4d33-97fa-7d300a101c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:09.630466 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c704a-15a0-4d33-97fa-7d300a101c69 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:09.630746 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c704a-15a0-4d33-97fa-7d300a101c69 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:09.631018 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c704a-15a0-4d33-97fa-7d300a101c69 None admin] PATH_INFO: `/v3/projects/e3148f82c47e4702b119cf867f5dcdcc` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:09.632222 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c704a-15a0-4d33-97fa-7d300a101c69 None admin] RBAC: Authorizing `identity:get_project(project_id=e3148f82c47e4702b119cf867f5dcdcc)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:09.634162 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c704a-15a0-4d33-97fa-7d300a101c69 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:09.634740 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8e570-1527-4250-825e-c3f547c6d7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:09.635583 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8e570-1527-4250-825e-c3f547c6d7cf None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:09.640345 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 86/177] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:09 2022] GET /identity/v3/projects/e3148f82c47e4702b119cf867f5dcdcc => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:09.640345 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c444b-7dae-405e-b447-6be11107eea4 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:09.640345 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c444b-7dae-405e-b447-6be11107eea4 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:09.640345 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c444b-7dae-405e-b447-6be11107eea4 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:09.640345 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 87/178] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:09.640345 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c444b-7dae-405e-b447-6be11107eea4 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:09.641192 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8e570-1527-4250-825e-c3f547c6d7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:09.641726 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8e570-1527-4250-825e-c3f547c6d7cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:09.716974 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:09.731950 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d8e570-1527-4250-825e-c3f547c6d7cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:09.734007 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d8e570-1527-4250-825e-c3f547c6d7cf None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:09.778763 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c444b-7dae-405e-b447-6be11107eea4 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:09.790906 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:09.814235 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fef45d-e9ed-435c-8ca9-92ac61b1d681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:09.814852 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fef45d-e9ed-435c-8ca9-92ac61b1d681 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:09.815127 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fef45d-e9ed-435c-8ca9-92ac61b1d681 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:09.815402 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fef45d-e9ed-435c-8ca9-92ac61b1d681 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:09.816164 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fef45d-e9ed-435c-8ca9-92ac61b1d681 None admin] RBAC: Adding query filter params (role.id=4207a743edc04b0aba9197fe3554b2e3, scope.project.id=e3148f82c47e4702b119cf867f5dcdcc, user.id=628f03c8b9b44e27bacc6919029346d1) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:09.816474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fef45d-e9ed-435c-8ca9-92ac61b1d681 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:09.817894 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fef45d-e9ed-435c-8ca9-92ac61b1d681 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:09.823654 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 88/179] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:14:09 2022] GET /identity/v3/role_assignments?user.id=628f03c8b9b44e27bacc6919029346d1&scope.project.id=e3148f82c47e4702b119cf867f5dcdcc&role.id=4207a743edc04b0aba9197fe3554b2e3 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:09.824552 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fef45d-e9ed-435c-8ca9-92ac61b1d681 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:09.840576 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 92/180] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:14:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:14:09.846804 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44078c3-e80c-4f35-94e9-86b3e5b2b36c None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:09.847148 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44078c3-e80c-4f35-94e9-86b3e5b2b36c None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:09.847401 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44078c3-e80c-4f35-94e9-86b3e5b2b36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:09.866556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-f44078c3-e80c-4f35-94e9-86b3e5b2b36c None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:09.868038 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-f44078c3-e80c-4f35-94e9-86b3e5b2b36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:09.950423 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 93/181] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:09.957523 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903a9e8-b858-4f7c-b37f-b38f7d5b8c6f None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:09.957819 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903a9e8-b858-4f7c-b37f-b38f7d5b8c6f None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:09.958028 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fef45d-e9ed-435c-8ca9-92ac61b1d681 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:09.958158 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903a9e8-b858-4f7c-b37f-b38f7d5b8c6f None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:09.959735 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 94/182] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:09 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:09.962335 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2970a-6607-49e9-96f0-5c0a8772d624 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:09.962624 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2970a-6607-49e9-96f0-5c0a8772d624 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:09.962897 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2970a-6607-49e9-96f0-5c0a8772d624 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:09.968975 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 95/183] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:09.968975 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2970a-6607-49e9-96f0-5c0a8772d624 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:09.974836 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:10.021206 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa67b3-3ebb-4907-80c6-b4a6945a8d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:10.022192 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa67b3-3ebb-4907-80c6-b4a6945a8d6b None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:10.022472 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa67b3-3ebb-4907-80c6-b4a6945a8d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:10.023051 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa67b3-3ebb-4907-80c6-b4a6945a8d6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:10.024530 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa67b3-3ebb-4907-80c6-b4a6945a8d6b None admin] RBAC: Adding query filter params (role.id=52b31061b72c45ccb76dd26bf31f8d3f, scope.project.id=e3148f82c47e4702b119cf867f5dcdcc, user.id=de65eb39f13f4a92a6a526fb5ab1f5c5) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:10.024874 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa67b3-3ebb-4907-80c6-b4a6945a8d6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:10.026362 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa67b3-3ebb-4907-80c6-b4a6945a8d6b None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:10.035480 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 89/184] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:14:09 2022] GET /identity/v3/role_assignments?user.id=de65eb39f13f4a92a6a526fb5ab1f5c5&scope.project.id=e3148f82c47e4702b119cf867f5dcdcc&role.id=52b31061b72c45ccb76dd26bf31f8d3f => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:10.035713 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa67b3-3ebb-4907-80c6-b4a6945a8d6b None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:10.075074 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:10.104214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2970a-6607-49e9-96f0-5c0a8772d624 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:10.158368 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022d316-e7ad-487f-8aa1-f6ee37eb975d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:10.159056 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022d316-e7ad-487f-8aa1-f6ee37eb975d None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:10.159335 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022d316-e7ad-487f-8aa1-f6ee37eb975d None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:10.159607 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022d316-e7ad-487f-8aa1-f6ee37eb975d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:10.172660 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6022d316-e7ad-487f-8aa1-f6ee37eb975d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:10.178065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-6022d316-e7ad-487f-8aa1-f6ee37eb975d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:14:10.178065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:10.178065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:10.178065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:10.178065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:10.178065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:10.178065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:10.178065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:10.178065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:10.178065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:10.178065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:10.178065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:10.178065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:10.178065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:10.178065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:10.178065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:10.179006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:10.179006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:10.179006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:10.179006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:10.179006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:10.179006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:10.179006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:10.179006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:10.179006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:10.179006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:10.179006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:10.179006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:10.179006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:10.179006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:10.179006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:10.179006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:10.179837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:10.179837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:10.179837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:10.179837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:10.179837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:10.179837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:10.179837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:10.179837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:10.179837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:10.179837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:10.179837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:14:10.179837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:10.179837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 96/185] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:14:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:10.181003 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff9530-7eb9-4a7d-ad64-7c57b7a6c419 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:10.181003 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff9530-7eb9-4a7d-ad64-7c57b7a6c419 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:10.181258 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff9530-7eb9-4a7d-ad64-7c57b7a6c419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:10.189154 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa67b3-3ebb-4907-80c6-b4a6945a8d6b None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:10.200549 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-21ff9530-7eb9-4a7d-ad64-7c57b7a6c419 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:10.201756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-21ff9530-7eb9-4a7d-ad64-7c57b7a6c419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:10.230458 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8809534-9d27-4a10-b007-499910d6de57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:10.231286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8809534-9d27-4a10-b007-499910d6de57 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:10.231566 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8809534-9d27-4a10-b007-499910d6de57 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:10.232026 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8809534-9d27-4a10-b007-499910d6de57 None admin] PATH_INFO: `/v3/projects/67f78794d6ab4968bc00d61a6015220c` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:10.233457 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8809534-9d27-4a10-b007-499910d6de57 None admin] RBAC: Authorizing `identity:get_project(project_id=67f78794d6ab4968bc00d61a6015220c)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:10.234914 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8809534-9d27-4a10-b007-499910d6de57 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:10.236639 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 90/186] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:10 2022] GET /identity/v3/projects/67f78794d6ab4968bc00d61a6015220c => generated 312 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:10.237984 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc56cfa-b25b-453a-87d8-c911e24cb44d None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:10.238263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc56cfa-b25b-453a-87d8-c911e24cb44d None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:10.238532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc56cfa-b25b-453a-87d8-c911e24cb44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:10.262514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-fcc56cfa-b25b-453a-87d8-c911e24cb44d None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:10.263722 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-fcc56cfa-b25b-453a-87d8-c911e24cb44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:10.288278 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 97/187] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:10.289346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff9530-7eb9-4a7d-ad64-7c57b7a6c419 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:10.296972 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:10.350790 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 91/188] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:10.357598 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc56cfa-b25b-453a-87d8-c911e24cb44d None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:10.369796 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:10.405106 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff9530-7eb9-4a7d-ad64-7c57b7a6c419 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:10.443869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0e3fd-4002-435c-9a23-8cf94a87b6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:10.444904 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0e3fd-4002-435c-9a23-8cf94a87b6e6 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:10.445666 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0e3fd-4002-435c-9a23-8cf94a87b6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:10.446359 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0e3fd-4002-435c-9a23-8cf94a87b6e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:10.447233 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0e3fd-4002-435c-9a23-8cf94a87b6e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:10.447881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0e3fd-4002-435c-9a23-8cf94a87b6e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:10.449606 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0e3fd-4002-435c-9a23-8cf94a87b6e6 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:10.465412 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 98/189] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:10.470048 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0e3fd-4002-435c-9a23-8cf94a87b6e6 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:10.501137 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc56cfa-b25b-453a-87d8-c911e24cb44d None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:10.543053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1eda2c-27e3-4c9c-82a5-039ab78cb92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:10.543688 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1eda2c-27e3-4c9c-82a5-039ab78cb92d None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:10.544057 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1eda2c-27e3-4c9c-82a5-039ab78cb92d None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:10.544328 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1eda2c-27e3-4c9c-82a5-039ab78cb92d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:10.555256 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1eda2c-27e3-4c9c-82a5-039ab78cb92d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:10.560280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-9c1eda2c-27e3-4c9c-82a5-039ab78cb92d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 04:14:10.560280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:10.560280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:10.560280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:10.560280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:10.560280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:10.560280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:10.560280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:10.560280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:10.560280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:10.560280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:10.560280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:10.560280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:10.560280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:10.560280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:10.560280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:10.561119 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:10.561119 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:10.561119 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:10.561119 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:10.561119 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:10.561119 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:10.561119 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:10.561119 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:10.561119 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:10.561119 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:10.561119 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:10.561119 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:10.561119 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:10.561119 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:10.561119 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:10.561119 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:10.561879 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:10.561879 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:10.561879 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:10.561879 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:10.561879 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:10.561879 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:10.561879 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:10.561879 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:10.561879 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:10.561879 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:10.561879 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 04:14:10.561879 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:10.561879 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 92/190] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Aug 23 04:14:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:10.562501 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1eda2c-27e3-4c9c-82a5-039ab78cb92d None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:10.588547 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0e3fd-4002-435c-9a23-8cf94a87b6e6 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:10.591936 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:10.625531 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e908862-3f5c-4351-b218-211a4a6178b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:10.626145 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e908862-3f5c-4351-b218-211a4a6178b5 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:10.626419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e908862-3f5c-4351-b218-211a4a6178b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:10.626687 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e908862-3f5c-4351-b218-211a4a6178b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:10.627414 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e908862-3f5c-4351-b218-211a4a6178b5 None admin] RBAC: Adding query filter params (role.id=aa03504b4fb642d288d07f4335e291e1, scope.project.id=67f78794d6ab4968bc00d61a6015220c, user.id=628f03c8b9b44e27bacc6919029346d1) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:10.627718 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e908862-3f5c-4351-b218-211a4a6178b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:10.629265 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e908862-3f5c-4351-b218-211a4a6178b5 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:10.631069 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:10.641406 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 99/191] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:14:10 2022] GET /identity/v3/role_assignments?user.id=628f03c8b9b44e27bacc6919029346d1&scope.project.id=67f78794d6ab4968bc00d61a6015220c&role.id=aa03504b4fb642d288d07f4335e291e1 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:10.643255 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e908862-3f5c-4351-b218-211a4a6178b5 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:10.682410 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1eda2c-27e3-4c9c-82a5-039ab78cb92d None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:10.725841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e456f7cb-d9ea-4313-9c06-1cdb946a343e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:10.726583 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456f7cb-d9ea-4313-9c06-1cdb946a343e None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:10.726956 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456f7cb-d9ea-4313-9c06-1cdb946a343e None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:10.727712 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456f7cb-d9ea-4313-9c06-1cdb946a343e None admin] PATH_INFO: `/v3/users/d81e03489837416d84b2b7a24728a933` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:10.743714 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e456f7cb-d9ea-4313-9c06-1cdb946a343e None admin] RBAC: Authorizing `identity:get_user(user_id=d81e03489837416d84b2b7a24728a933)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:10.745365 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e456f7cb-d9ea-4313-9c06-1cdb946a343e None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:10.747350 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 93/192] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:14:10 2022] GET /identity/v3/users/d81e03489837416d84b2b7a24728a933 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:10.748053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e456f7cb-d9ea-4313-9c06-1cdb946a343e None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:10.772220 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e908862-3f5c-4351-b218-211a4a6178b5 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:10.819781 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea2d0b-4567-40ea-a18a-671bae7e269e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:10.823291 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea2d0b-4567-40ea-a18a-671bae7e269e None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:10.823291 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea2d0b-4567-40ea-a18a-671bae7e269e None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:10.823291 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea2d0b-4567-40ea-a18a-671bae7e269e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:10.823291 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea2d0b-4567-40ea-a18a-671bae7e269e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:10.824452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea2d0b-4567-40ea-a18a-671bae7e269e None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:10.829792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 100/193] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:14:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:10.829792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea2d0b-4567-40ea-a18a-671bae7e269e None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:10.860643 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:10.882978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e456f7cb-d9ea-4313-9c06-1cdb946a343e None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:10.907736 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:10.917198 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dddd12-06e1-4a13-88f6-b8d3e7fd88c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:10.917792 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dddd12-06e1-4a13-88f6-b8d3e7fd88c9 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:10.918068 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dddd12-06e1-4a13-88f6-b8d3e7fd88c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:10.918339 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dddd12-06e1-4a13-88f6-b8d3e7fd88c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:10.919013 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dddd12-06e1-4a13-88f6-b8d3e7fd88c9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:10.919314 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dddd12-06e1-4a13-88f6-b8d3e7fd88c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:10.920773 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dddd12-06e1-4a13-88f6-b8d3e7fd88c9 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:10.929147 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 94/194] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:14:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:10.930000 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dddd12-06e1-4a13-88f6-b8d3e7fd88c9 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:10.944742 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea2d0b-4567-40ea-a18a-671bae7e269e None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:10.978445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f1b9a8-36d1-4b1a-9c91-8fe2af1581f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:10.979079 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1b9a8-36d1-4b1a-9c91-8fe2af1581f5 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:10.979359 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1b9a8-36d1-4b1a-9c91-8fe2af1581f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:10.979629 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1b9a8-36d1-4b1a-9c91-8fe2af1581f5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:10.989545 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f1b9a8-36d1-4b1a-9c91-8fe2af1581f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:10.996204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-a6f1b9a8-36d1-4b1a-9c91-8fe2af1581f5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 04:14:10.996204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:10.996204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:10.996204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:10.996204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:10.996204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:10.996204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:10.996204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:10.996204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:10.996204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:10.996204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:10.996204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:10.996204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:10.996204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:10.996204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:10.996204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:10.997214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:10.997214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:10.997214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:10.997214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:10.997214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:10.997214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:10.997214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:10.997214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:10.997214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:10.997214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:10.997214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:10.997214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:10.997214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:10.997214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:10.997214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:10.997214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:10.997988 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:10.997988 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:10.997988 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:10.997988 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:10.997988 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:10.997988 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:10.997988 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:10.997988 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:10.997988 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:10.997988 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:10.997988 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 04:14:10.997988 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:10.997988 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 101/195] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:10.998647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f1b9a8-36d1-4b1a-9c91-8fe2af1581f5 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:11.032416 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dddd12-06e1-4a13-88f6-b8d3e7fd88c9 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:11.064230 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8fe93-caf7-4ae7-80cf-36fefaa96396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:11.064835 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8fe93-caf7-4ae7-80cf-36fefaa96396 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:11.065108 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8fe93-caf7-4ae7-80cf-36fefaa96396 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:11.065375 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8fe93-caf7-4ae7-80cf-36fefaa96396 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:11.083904 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:11.098030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f1b9a8-36d1-4b1a-9c91-8fe2af1581f5 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:11.123335 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:11.132549 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facf947a-2bba-4c53-8cb0-d2c9c936c2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:11.133268 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf947a-2bba-4c53-8cb0-d2c9c936c2a8 None admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:11.133553 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf947a-2bba-4c53-8cb0-d2c9c936c2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:11.133826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf947a-2bba-4c53-8cb0-d2c9c936c2a8 None admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:11.134473 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facf947a-2bba-4c53-8cb0-d2c9c936c2a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:11.135115 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c8fe93-caf7-4ae7-80cf-36fefaa96396 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:11.135987 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facf947a-2bba-4c53-8cb0-d2c9c936c2a8 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:11.136606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c8fe93-caf7-4ae7-80cf-36fefaa96396 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:11.158200 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 102/196] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Aug 23 04:14:10 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:14:11.158200 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facf947a-2bba-4c53-8cb0-d2c9c936c2a8 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:11.270408 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 95/197] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:14:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:14:11.271093 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8fe93-caf7-4ae7-80cf-36fefaa96396 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:11.293528 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facf947a-2bba-4c53-8cb0-d2c9c936c2a8 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:11.332188 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2110d0-51fd-4f32-afbb-f885ffa656f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:11.332842 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2110d0-51fd-4f32-afbb-f885ffa656f4 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:11.333108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2110d0-51fd-4f32-afbb-f885ffa656f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:11.333378 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2110d0-51fd-4f32-afbb-f885ffa656f4 None admin] PATH_INFO: `/v3/users/d81e03489837416d84b2b7a24728a933` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:11.334582 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2110d0-51fd-4f32-afbb-f885ffa656f4 None admin] RBAC: Authorizing `identity:get_user(user_id=d81e03489837416d84b2b7a24728a933)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:11.336780 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2110d0-51fd-4f32-afbb-f885ffa656f4 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:11.337494 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:11.338473 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 103/198] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:14:11 2022] GET /identity/v3/users/d81e03489837416d84b2b7a24728a933 => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:11.340043 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2110d0-51fd-4f32-afbb-f885ffa656f4 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:11.400826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:11.413679 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8fe93-caf7-4ae7-80cf-36fefaa96396 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:11.461711 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0b1d4-7a88-4a29-b47d-76026a66fa68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:11.462340 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0b1d4-7a88-4a29-b47d-76026a66fa68 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:11.462614 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0b1d4-7a88-4a29-b47d-76026a66fa68 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:11.462880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0b1d4-7a88-4a29-b47d-76026a66fa68 None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:11.463559 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d0b1d4-7a88-4a29-b47d-76026a66fa68 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:11.464253 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d0b1d4-7a88-4a29-b47d-76026a66fa68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:11.465640 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d0b1d4-7a88-4a29-b47d-76026a66fa68 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:11.476183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 96/199] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Aug 23 04:14:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:11.477178 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0b1d4-7a88-4a29-b47d-76026a66fa68 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:11.478468 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2110d0-51fd-4f32-afbb-f885ffa656f4 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:11.525288 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c7650-b33c-4238-90c2-b6dc2c30847e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:11.525939 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c7650-b33c-4238-90c2-b6dc2c30847e None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:11.526282 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c7650-b33c-4238-90c2-b6dc2c30847e None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:11.526560 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c7650-b33c-4238-90c2-b6dc2c30847e None admin] PATH_INFO: `/v3/projects/95a15538272e426ba99c0b50e2f30f52` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:11.528225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c7650-b33c-4238-90c2-b6dc2c30847e None admin] RBAC: Authorizing `identity:get_project(project_id=95a15538272e426ba99c0b50e2f30f52)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:11.529688 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c7650-b33c-4238-90c2-b6dc2c30847e None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:11.535937 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 104/200] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:11 2022] GET /identity/v3/projects/95a15538272e426ba99c0b50e2f30f52 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:11.535937 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c7650-b33c-4238-90c2-b6dc2c30847e None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:11.602711 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0b1d4-7a88-4a29-b47d-76026a66fa68 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:11.614556 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:11.653399 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b404a69-13a3-4d58-ad1d-47930cfe1bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:11.654498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b404a69-13a3-4d58-ad1d-47930cfe1bd9 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:11.655038 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b404a69-13a3-4d58-ad1d-47930cfe1bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:11.655316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b404a69-13a3-4d58-ad1d-47930cfe1bd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:11.657155 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c7650-b33c-4238-90c2-b6dc2c30847e None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:11.677294 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:11.696908 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937cadbf-f537-48c2-b030-3a1ad2f8dded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:11.697806 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937cadbf-f537-48c2-b030-3a1ad2f8dded None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:11.698219 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937cadbf-f537-48c2-b030-3a1ad2f8dded None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:11.698634 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937cadbf-f537-48c2-b030-3a1ad2f8dded None admin] PATH_INFO: `/v3/users/28038a4cdebf4eea929542b042210018` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:11.700061 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937cadbf-f537-48c2-b030-3a1ad2f8dded None admin] RBAC: Authorizing `identity:get_user(user_id=28038a4cdebf4eea929542b042210018)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:11.701756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937cadbf-f537-48c2-b030-3a1ad2f8dded None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:11.703625 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 105/201] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:14:11 2022] GET /identity/v3/users/28038a4cdebf4eea929542b042210018 => generated 289 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:11.708904 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937cadbf-f537-48c2-b030-3a1ad2f8dded None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:11.758801 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b404a69-13a3-4d58-ad1d-47930cfe1bd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:11.760410 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b404a69-13a3-4d58-ad1d-47930cfe1bd9 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:11.830227 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937cadbf-f537-48c2-b030-3a1ad2f8dded None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:11.871762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c74d82-6a80-4bf4-8899-9778ded608cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:11.872636 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c74d82-6a80-4bf4-8899-9778ded608cb None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:11.872983 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c74d82-6a80-4bf4-8899-9778ded608cb None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:11.873321 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c74d82-6a80-4bf4-8899-9778ded608cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:11.874124 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c74d82-6a80-4bf4-8899-9778ded608cb None admin] RBAC: Adding query filter params (role.id=52b31061b72c45ccb76dd26bf31f8d3f, scope.project.id=95a15538272e426ba99c0b50e2f30f52, user.id=d81e03489837416d84b2b7a24728a933) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:11.874494 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c74d82-6a80-4bf4-8899-9778ded608cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:11.876375 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c74d82-6a80-4bf4-8899-9778ded608cb None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:11.882739 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 106/202] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:14:11 2022] GET /identity/v3/role_assignments?user.id=d81e03489837416d84b2b7a24728a933&scope.project.id=95a15538272e426ba99c0b50e2f30f52&role.id=52b31061b72c45ccb76dd26bf31f8d3f => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:11.886196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 97/203] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:14:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:14:11.889004 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c74d82-6a80-4bf4-8899-9778ded608cb None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:11.894231 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b404a69-13a3-4d58-ad1d-47930cfe1bd9 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:11.906045 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:12.015192 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:12.022222 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b404a69-13a3-4d58-ad1d-47930cfe1bd9 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:12.070311 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcf097-086a-4231-ba72-30d914622539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:12.071041 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcf097-086a-4231-ba72-30d914622539 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:12.071427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcf097-086a-4231-ba72-30d914622539 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:12.071705 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcf097-086a-4231-ba72-30d914622539 None admin] PATH_INFO: `/v3/projects/95a15538272e426ba99c0b50e2f30f52` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:12.074672 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adcf097-086a-4231-ba72-30d914622539 None admin] RBAC: Authorizing `identity:get_project(project_id=95a15538272e426ba99c0b50e2f30f52)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:12.076372 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adcf097-086a-4231-ba72-30d914622539 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:12.078291 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 98/204] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:11 2022] GET /identity/v3/projects/95a15538272e426ba99c0b50e2f30f52 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:12.087165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcf097-086a-4231-ba72-30d914622539 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:12.104782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c74d82-6a80-4bf4-8899-9778ded608cb None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:12.151401 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a4856-58d6-4b93-808d-577cf88fad8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:12.153037 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a4856-58d6-4b93-808d-577cf88fad8c None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:12.153524 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a4856-58d6-4b93-808d-577cf88fad8c None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:12.153994 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a4856-58d6-4b93-808d-577cf88fad8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:12.203701 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:12.247866 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcf097-086a-4231-ba72-30d914622539 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:12.288061 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a4856-58d6-4b93-808d-577cf88fad8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:12.290292 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a4856-58d6-4b93-808d-577cf88fad8c None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:12.293941 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8a922-5dd8-4b3c-ab35-7ea34e42eab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:12.294566 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8a922-5dd8-4b3c-ab35-7ea34e42eab5 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:12.294838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8a922-5dd8-4b3c-ab35-7ea34e42eab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:12.295105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8a922-5dd8-4b3c-ab35-7ea34e42eab5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:12.295840 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8a922-5dd8-4b3c-ab35-7ea34e42eab5 None admin] RBAC: Adding query filter params (role.id=4207a743edc04b0aba9197fe3554b2e3, scope.project.id=95a15538272e426ba99c0b50e2f30f52, user.id=d81e03489837416d84b2b7a24728a933) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:12.296168 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8a922-5dd8-4b3c-ab35-7ea34e42eab5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:12.297582 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8a922-5dd8-4b3c-ab35-7ea34e42eab5 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:12.309939 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 99/205] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:14:12 2022] GET /identity/v3/role_assignments?user.id=d81e03489837416d84b2b7a24728a933&scope.project.id=95a15538272e426ba99c0b50e2f30f52&role.id=4207a743edc04b0aba9197fe3554b2e3 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:14:12.389336 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:12.468344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 107/206] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:14:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:14:12.474115 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8a922-5dd8-4b3c-ab35-7ea34e42eab5 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:12.617542 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8a922-5dd8-4b3c-ab35-7ea34e42eab5 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:12.669158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10f369-2e54-4133-aef4-fca02e2f53f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:12.670214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10f369-2e54-4133-aef4-fca02e2f53f7 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:12.670691 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10f369-2e54-4133-aef4-fca02e2f53f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:12.671172 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10f369-2e54-4133-aef4-fca02e2f53f7 None admin] PATH_INFO: `/v3/projects/e3148f82c47e4702b119cf867f5dcdcc` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:12.676396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10f369-2e54-4133-aef4-fca02e2f53f7 None admin] RBAC: Authorizing `identity:get_project(project_id=e3148f82c47e4702b119cf867f5dcdcc)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:12.678079 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10f369-2e54-4133-aef4-fca02e2f53f7 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:12.679998 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 100/207] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:12 2022] GET /identity/v3/projects/e3148f82c47e4702b119cf867f5dcdcc => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:12.686205 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a4856-58d6-4b93-808d-577cf88fad8c None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:12.797932 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a4856-58d6-4b93-808d-577cf88fad8c None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:12.835380 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4188f2-dd1e-4d14-be11-65e40aca8116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:12.836480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4188f2-dd1e-4d14-be11-65e40aca8116 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:12.836802 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4188f2-dd1e-4d14-be11-65e40aca8116 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:12.837072 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4188f2-dd1e-4d14-be11-65e40aca8116 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:12.837831 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4188f2-dd1e-4d14-be11-65e40aca8116 None admin] RBAC: Adding query filter params (role.id=4be260163ba342e9b42232a49eec9fec, scope.project.id=e3148f82c47e4702b119cf867f5dcdcc, user.id=28038a4cdebf4eea929542b042210018) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:12.838396 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4188f2-dd1e-4d14-be11-65e40aca8116 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:12.840202 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4188f2-dd1e-4d14-be11-65e40aca8116 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:12.846826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 108/208] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:14:12 2022] GET /identity/v3/role_assignments?user.id=28038a4cdebf4eea929542b042210018&scope.project.id=e3148f82c47e4702b119cf867f5dcdcc&role.id=4be260163ba342e9b42232a49eec9fec => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:13.479653 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e563b-2032-4c3f-bfe9-9e00e85b3b02 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:13.480194 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e563b-2032-4c3f-bfe9-9e00e85b3b02 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:13.480771 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e563b-2032-4c3f-bfe9-9e00e85b3b02 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:13.482298 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 101/209] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:13.489651 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f31bb2-ca6b-4c79-9286-4f6dec2e4485 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:13.490183 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f31bb2-ca6b-4c79-9286-4f6dec2e4485 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:13.490413 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f31bb2-ca6b-4c79-9286-4f6dec2e4485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:13.517005 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-a7f31bb2-ca6b-4c79-9286-4f6dec2e4485 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:13.518319 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-a7f31bb2-ca6b-4c79-9286-4f6dec2e4485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:13.526772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1657779-e08d-413c-b0e7-177939452b10 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:13.527099 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1657779-e08d-413c-b0e7-177939452b10 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:13.527336 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1657779-e08d-413c-b0e7-177939452b10 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:13.528876 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 102/210] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:14:13.531280 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:13.535905 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df776883-a636-48cd-ba60-92caf203926d None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:13.536220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df776883-a636-48cd-ba60-92caf203926d None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:13.536499 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df776883-a636-48cd-ba60-92caf203926d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:13.563866 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-df776883-a636-48cd-ba60-92caf203926d None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:13.565310 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-df776883-a636-48cd-ba60-92caf203926d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:13.568896 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:13.613007 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 109/211] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:13.660763 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 103/212] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:13.890032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f31bb2-ca6b-4c79-9286-4f6dec2e4485 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:13.995715 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df776883-a636-48cd-ba60-92caf203926d None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:14.029036 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f31bb2-ca6b-4c79-9286-4f6dec2e4485 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:14.076253 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be1013b-d6c3-4e0b-b815-b941fc6df3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:14.077241 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1013b-d6c3-4e0b-b815-b941fc6df3cc None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:14.077617 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1013b-d6c3-4e0b-b815-b941fc6df3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:14.077993 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1013b-d6c3-4e0b-b815-b941fc6df3cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:14.092353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be1013b-d6c3-4e0b-b815-b941fc6df3cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:14.098964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-2be1013b-d6c3-4e0b-b815-b941fc6df3cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:14.098964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:14.098964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:14.098964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:14.098964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:14.098964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:14.098964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:14.098964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:14.098964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:14.098964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:14.098964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:14.098964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:14.098964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:14.098964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:14.098964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:14.098964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:14.100662 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:14.100662 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:14.100662 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:14.100662 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:14.100662 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:14.100662 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:14.100662 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:14.100662 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:14.100662 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:14.100662 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:14.100662 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:14.100662 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:14.100662 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:14.100662 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:14.100662 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:14.100662 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:14.101701 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:14.101701 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:14.101701 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:14.101701 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:14.101701 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:14.101701 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:14.101701 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:14.101701 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:14.101701 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:14.101701 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:14.101701 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:14.101701 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:14.101701 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 110/213] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:14.106784 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be1013b-d6c3-4e0b-b815-b941fc6df3cc None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:14.112986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:14.139889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df776883-a636-48cd-ba60-92caf203926d None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:14.181975 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f241f1-87ba-4863-a47c-4e10e01c9b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:14.182627 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f241f1-87ba-4863-a47c-4e10e01c9b56 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:14.185007 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f241f1-87ba-4863-a47c-4e10e01c9b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:14.185007 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f241f1-87ba-4863-a47c-4e10e01c9b56 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:14.185007 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f241f1-87ba-4863-a47c-4e10e01c9b56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:14.187240 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f241f1-87ba-4863-a47c-4e10e01c9b56 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:14.187533 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 104/214] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:14:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:14:14.192185 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55addd59-21c0-4a80-8929-cbdc29f1c4f0 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:14.192464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55addd59-21c0-4a80-8929-cbdc29f1c4f0 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:14.192732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55addd59-21c0-4a80-8929-cbdc29f1c4f0 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:14.194155 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 105/215] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:14.194698 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55addd59-21c0-4a80-8929-cbdc29f1c4f0 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:14.211381 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:14.237780 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be1013b-d6c3-4e0b-b815-b941fc6df3cc None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:14.284550 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670578d1-c8d2-44e6-aa03-d9a321c25685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:14.285471 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670578d1-c8d2-44e6-aa03-d9a321c25685 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:14.285944 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670578d1-c8d2-44e6-aa03-d9a321c25685 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:14.286398 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670578d1-c8d2-44e6-aa03-d9a321c25685 None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:14.287243 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670578d1-c8d2-44e6-aa03-d9a321c25685 None admin] RBAC: Adding query filter params (name=member) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:14.287714 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670578d1-c8d2-44e6-aa03-d9a321c25685 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:14.289944 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670578d1-c8d2-44e6-aa03-d9a321c25685 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:14.301955 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 111/216] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Aug 23 04:14:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:14.303550 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8385c9-cc97-4b22-8b9c-80df559aac8b None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:14.304062 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8385c9-cc97-4b22-8b9c-80df559aac8b None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:14.304564 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8385c9-cc97-4b22-8b9c-80df559aac8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:14.322449 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55addd59-21c0-4a80-8929-cbdc29f1c4f0 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:14.328083 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-1a8385c9-cc97-4b22-8b9c-80df559aac8b None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:14.329507 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-1a8385c9-cc97-4b22-8b9c-80df559aac8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:14.365073 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6544d634-5314-4f81-8ecb-c0aee22eba06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:14.365689 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544d634-5314-4f81-8ecb-c0aee22eba06 None admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:14.365959 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544d634-5314-4f81-8ecb-c0aee22eba06 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:14.366225 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544d634-5314-4f81-8ecb-c0aee22eba06 None admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:14.366941 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6544d634-5314-4f81-8ecb-c0aee22eba06 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:14.368411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6544d634-5314-4f81-8ecb-c0aee22eba06 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:14.392914 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 106/217] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Aug 23 04:14:14 2022] POST /identity/v3/users => generated 297 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:14:14.393759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6544d634-5314-4f81-8ecb-c0aee22eba06 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:14.408488 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:14.418913 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 112/218] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:14.569866 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:14.622371 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6544d634-5314-4f81-8ecb-c0aee22eba06 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:14.660609 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae76dc7-de95-4a28-9a06-b2d8d3e9a400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:14.661252 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae76dc7-de95-4a28-9a06-b2d8d3e9a400 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:14.661793 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae76dc7-de95-4a28-9a06-b2d8d3e9a400 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:14.661793 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae76dc7-de95-4a28-9a06-b2d8d3e9a400 None admin] PATH_INFO: `/v3/users/4772005528a1418a904ae1d87d9e0f18` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:14.683865 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662d624-88ae-48ac-bc81-a5cee370ceea None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:14.684821 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662d624-88ae-48ac-bc81-a5cee370ceea None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:14.689351 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662d624-88ae-48ac-bc81-a5cee370ceea None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:14.689351 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 113/219] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:14.690160 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae76dc7-de95-4a28-9a06-b2d8d3e9a400 None admin] RBAC: Authorizing `identity:get_user(user_id=4772005528a1418a904ae1d87d9e0f18)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:14.691655 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae76dc7-de95-4a28-9a06-b2d8d3e9a400 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:14.694072 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 107/220] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:14:14 2022] GET /identity/v3/users/4772005528a1418a904ae1d87d9e0f18 => generated 297 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:14.695504 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcbcd0-6974-42a0-8d6c-8708ce47908c None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:14.695787 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcbcd0-6974-42a0-8d6c-8708ce47908c None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:14.696267 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcbcd0-6974-42a0-8d6c-8708ce47908c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:14.700001 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae76dc7-de95-4a28-9a06-b2d8d3e9a400 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:14.725109 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-25dcbcd0-6974-42a0-8d6c-8708ce47908c None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:14.725545 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-25dcbcd0-6974-42a0-8d6c-8708ce47908c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:14.829934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae76dc7-de95-4a28-9a06-b2d8d3e9a400 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:14.835810 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 114/221] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:14.836555 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dcbcd0-6974-42a0-8d6c-8708ce47908c None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:14.861191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:14.878435 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a654546-e5ce-440d-b76c-c5c6e75db0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:14.879062 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a654546-e5ce-440d-b76c-c5c6e75db0ec None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:14.879341 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a654546-e5ce-440d-b76c-c5c6e75db0ec None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:14.879616 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a654546-e5ce-440d-b76c-c5c6e75db0ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:14.958950 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a654546-e5ce-440d-b76c-c5c6e75db0ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:14.960474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a654546-e5ce-440d-b76c-c5c6e75db0ec None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:14.962187 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dcbcd0-6974-42a0-8d6c-8708ce47908c None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:15.016446 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:15.034999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ea576-f9e5-4c8b-ba68-323c8a9d435c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:15.035652 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ea576-f9e5-4c8b-ba68-323c8a9d435c None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:15.035942 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ea576-f9e5-4c8b-ba68-323c8a9d435c None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:15.036224 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ea576-f9e5-4c8b-ba68-323c8a9d435c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:15.058429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50ea576-f9e5-4c8b-ba68-323c8a9d435c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:15.063151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-c50ea576-f9e5-4c8b-ba68-323c8a9d435c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:14:15.063151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:15.063151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:15.063151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:15.063151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:15.063151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:15.063151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:15.063151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:15.063151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:15.063151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:15.063151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:15.063151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:15.063151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:15.063151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:15.063151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:15.063151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:15.064013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:15.064013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:15.064013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:15.064013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:15.064013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:15.064013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:15.064013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:15.064013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:15.064013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:15.064013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:15.064013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:15.064013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:15.064013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:15.064013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:15.064013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:15.064013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:15.064788 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:15.064788 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:15.064788 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:15.064788 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:15.064788 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:15.064788 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:15.064788 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:15.064788 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:15.064788 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:15.064788 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:15.064788 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:14:15.064788 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:15.064788 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 115/222] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:14:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:15.071780 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ea576-f9e5-4c8b-ba68-323c8a9d435c None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:15.127843 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 108/223] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:14:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:14:15.134309 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a654546-e5ce-440d-b76c-c5c6e75db0ec None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:15.163268 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:15.225595 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ea576-f9e5-4c8b-ba68-323c8a9d435c None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:15.261204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79fd74-73d9-46ee-ac1f-9698a7f5428b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:15.262006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79fd74-73d9-46ee-ac1f-9698a7f5428b None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:15.262386 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79fd74-73d9-46ee-ac1f-9698a7f5428b None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:15.262755 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79fd74-73d9-46ee-ac1f-9698a7f5428b None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:15.263531 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f79fd74-73d9-46ee-ac1f-9698a7f5428b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:15.263954 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f79fd74-73d9-46ee-ac1f-9698a7f5428b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:15.265458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f79fd74-73d9-46ee-ac1f-9698a7f5428b None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:15.267876 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a654546-e5ce-440d-b76c-c5c6e75db0ec None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:15.274031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 116/224] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:15.274964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79fd74-73d9-46ee-ac1f-9698a7f5428b None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:15.312760 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72f9a0-9da3-4bdb-a347-a75d4fba331a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:15.313692 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72f9a0-9da3-4bdb-a347-a75d4fba331a None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:15.314115 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72f9a0-9da3-4bdb-a347-a75d4fba331a None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:15.314531 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72f9a0-9da3-4bdb-a347-a75d4fba331a None admin] PATH_INFO: `/v3/projects/95a15538272e426ba99c0b50e2f30f52` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:15.316329 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72f9a0-9da3-4bdb-a347-a75d4fba331a None admin] RBAC: Authorizing `identity:get_project(project_id=95a15538272e426ba99c0b50e2f30f52)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:15.318359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72f9a0-9da3-4bdb-a347-a75d4fba331a None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:15.326088 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 109/225] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:15 2022] GET /identity/v3/projects/95a15538272e426ba99c0b50e2f30f52 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:15.326088 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72f9a0-9da3-4bdb-a347-a75d4fba331a None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:15.337828 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:15.393447 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79fd74-73d9-46ee-ac1f-9698a7f5428b None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:15.441373 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:15.444110 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaaad7b-db2b-4ed4-8b57-86bc1ad6ba7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:15.445537 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaaad7b-db2b-4ed4-8b57-86bc1ad6ba7a None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:15.447393 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaaad7b-db2b-4ed4-8b57-86bc1ad6ba7a None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:15.448170 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaaad7b-db2b-4ed4-8b57-86bc1ad6ba7a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:15.454319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72f9a0-9da3-4bdb-a347-a75d4fba331a None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:15.475580 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaaad7b-db2b-4ed4-8b57-86bc1ad6ba7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:15.483381 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-cfaaad7b-db2b-4ed4-8b57-86bc1ad6ba7a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 04:14:15.483381 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:15.483381 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:15.483381 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:15.483381 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:15.483381 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:15.483381 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:15.483381 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:15.483381 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:15.483381 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:15.483381 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:15.483381 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:15.483381 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:15.483381 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:15.483381 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:15.483381 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:15.484306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:15.484306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:15.484306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:15.484306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:15.484306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:15.484306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:15.484306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:15.484306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:15.484306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:15.484306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:15.484306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:15.484306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:15.484306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:15.484306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:15.484306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:15.484306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:15.485103 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:15.485103 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:15.485103 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:15.485103 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:15.485103 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:15.485103 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:15.485103 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:15.485103 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:15.485103 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:15.485103 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:15.485103 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 04:14:15.485103 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:15.485696 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 117/226] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Aug 23 04:14:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:15.486312 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaaad7b-db2b-4ed4-8b57-86bc1ad6ba7a None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:15.498847 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e202ff7d-e890-47c0-9667-e0611b2fdef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:15.499819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202ff7d-e890-47c0-9667-e0611b2fdef8 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:15.500360 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202ff7d-e890-47c0-9667-e0611b2fdef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:15.500842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202ff7d-e890-47c0-9667-e0611b2fdef8 None admin] PATH_INFO: `/v3/users/d81e03489837416d84b2b7a24728a933` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:15.502244 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e202ff7d-e890-47c0-9667-e0611b2fdef8 None admin] RBAC: Authorizing `identity:get_user(user_id=d81e03489837416d84b2b7a24728a933)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:15.504188 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e202ff7d-e890-47c0-9667-e0611b2fdef8 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:15.505973 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 110/227] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:14:15 2022] GET /identity/v3/users/d81e03489837416d84b2b7a24728a933 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:15.507028 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e202ff7d-e890-47c0-9667-e0611b2fdef8 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:15.593680 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaaad7b-db2b-4ed4-8b57-86bc1ad6ba7a None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:15.603036 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:15.621164 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e202ff7d-e890-47c0-9667-e0611b2fdef8 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:15.630242 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c55f6-9726-4648-a274-6d2962924298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:15.630970 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c55f6-9726-4648-a274-6d2962924298 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:15.631368 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c55f6-9726-4648-a274-6d2962924298 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:15.631743 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c55f6-9726-4648-a274-6d2962924298 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:15.632689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29c55f6-9726-4648-a274-6d2962924298 None admin] RBAC: Adding query filter params (role.id=aa03504b4fb642d288d07f4335e291e1, scope.project.id=95a15538272e426ba99c0b50e2f30f52, user.id=4772005528a1418a904ae1d87d9e0f18) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:15.633084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29c55f6-9726-4648-a274-6d2962924298 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:15.634803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29c55f6-9726-4648-a274-6d2962924298 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:15.643946 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 118/228] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:14:15 2022] GET /identity/v3/role_assignments?user.id=4772005528a1418a904ae1d87d9e0f18&scope.project.id=95a15538272e426ba99c0b50e2f30f52&role.id=aa03504b4fb642d288d07f4335e291e1 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:15.643946 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c55f6-9726-4648-a274-6d2962924298 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:15.678195 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41055c1-b38e-4a92-b5c9-f8ba27a544ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:15.679680 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41055c1-b38e-4a92-b5c9-f8ba27a544ef None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:15.680375 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41055c1-b38e-4a92-b5c9-f8ba27a544ef None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:15.680941 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41055c1-b38e-4a92-b5c9-f8ba27a544ef None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:15.681861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41055c1-b38e-4a92-b5c9-f8ba27a544ef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:15.682394 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41055c1-b38e-4a92-b5c9-f8ba27a544ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:15.684240 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41055c1-b38e-4a92-b5c9-f8ba27a544ef None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:15.696340 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 111/229] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:14:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:15.704536 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41055c1-b38e-4a92-b5c9-f8ba27a544ef None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:15.772135 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c55f6-9726-4648-a274-6d2962924298 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:15.777299 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:15.813674 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134050f-56fe-4e90-9d5c-e55edb128919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:15.814339 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134050f-56fe-4e90-9d5c-e55edb128919 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:15.814627 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134050f-56fe-4e90-9d5c-e55edb128919 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:15.814898 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134050f-56fe-4e90-9d5c-e55edb128919 None admin] PATH_INFO: `/v3/projects/95a15538272e426ba99c0b50e2f30f52` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:15.817253 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4134050f-56fe-4e90-9d5c-e55edb128919 None admin] RBAC: Authorizing `identity:get_project(project_id=95a15538272e426ba99c0b50e2f30f52)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:15.818655 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4134050f-56fe-4e90-9d5c-e55edb128919 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:15.826035 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 119/230] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:15 2022] GET /identity/v3/projects/95a15538272e426ba99c0b50e2f30f52 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:15.832072 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134050f-56fe-4e90-9d5c-e55edb128919 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:15.895726 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:15.921144 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41055c1-b38e-4a92-b5c9-f8ba27a544ef None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:15.959158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e9d0d-470a-4535-8e39-1862522981fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:15.959804 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e9d0d-470a-4535-8e39-1862522981fa None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:15.960117 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e9d0d-470a-4535-8e39-1862522981fa None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:15.960390 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e9d0d-470a-4535-8e39-1862522981fa None admin] PATH_INFO: `/v3/users/d81e03489837416d84b2b7a24728a933` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:15.961904 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9e9d0d-470a-4535-8e39-1862522981fa None admin] RBAC: Authorizing `identity:get_user(user_id=d81e03489837416d84b2b7a24728a933)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:15.964396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9e9d0d-470a-4535-8e39-1862522981fa None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:15.966340 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 112/231] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:14:15 2022] GET /identity/v3/users/d81e03489837416d84b2b7a24728a933 => generated 283 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:15.972215 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e9d0d-470a-4535-8e39-1862522981fa None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:16.019184 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134050f-56fe-4e90-9d5c-e55edb128919 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:16.086030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53553106-445c-40b4-b941-8cd5409ed0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:16.086842 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53553106-445c-40b4-b941-8cd5409ed0c1 None admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:16.087128 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53553106-445c-40b4-b941-8cd5409ed0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:16.087409 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53553106-445c-40b4-b941-8cd5409ed0c1 None admin] PATH_INFO: `/v3/projects/95a15538272e426ba99c0b50e2f30f52/users/d81e03489837416d84b2b7a24728a933/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:16.093237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53553106-445c-40b4-b941-8cd5409ed0c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=95a15538272e426ba99c0b50e2f30f52, user_id=d81e03489837416d84b2b7a24728a933, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:16.094734 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53553106-445c-40b4-b941-8cd5409ed0c1 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:16.120664 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 120/232] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Aug 23 04:14:15 2022] PUT /identity/v3/projects/95a15538272e426ba99c0b50e2f30f52/users/d81e03489837416d84b2b7a24728a933/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:14:16.137488 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:16.152650 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e9d0d-470a-4535-8e39-1862522981fa None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:16.190808 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5dac72-c167-4a26-b3f2-e31f5350d49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:16.191805 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5dac72-c167-4a26-b3f2-e31f5350d49b None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:16.192345 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5dac72-c167-4a26-b3f2-e31f5350d49b None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:16.192812 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5dac72-c167-4a26-b3f2-e31f5350d49b None admin] PATH_INFO: `/v3/projects/95a15538272e426ba99c0b50e2f30f52` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:16.194158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5dac72-c167-4a26-b3f2-e31f5350d49b None admin] RBAC: Authorizing `identity:get_project(project_id=95a15538272e426ba99c0b50e2f30f52)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:16.195873 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5dac72-c167-4a26-b3f2-e31f5350d49b None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:16.197977 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 113/233] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:15 2022] GET /identity/v3/projects/95a15538272e426ba99c0b50e2f30f52 => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:14:16.204023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53553106-445c-40b4-b941-8cd5409ed0c1 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:16.359432 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:16.362857 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53553106-445c-40b4-b941-8cd5409ed0c1 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:16.400771 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfac7d8-a4a1-47d1-b044-c3613bc6df2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:16.401442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfac7d8-a4a1-47d1-b044-c3613bc6df2d None admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:16.401716 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfac7d8-a4a1-47d1-b044-c3613bc6df2d None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:16.401985 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfac7d8-a4a1-47d1-b044-c3613bc6df2d None admin] PATH_INFO: `/v3/projects/95a15538272e426ba99c0b50e2f30f52/users/d81e03489837416d84b2b7a24728a933/roles/4207a743edc04b0aba9197fe3554b2e3` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:16.404243 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfac7d8-a4a1-47d1-b044-c3613bc6df2d None admin] RBAC: Authorizing `identity:create_grant(project_id=95a15538272e426ba99c0b50e2f30f52, user_id=d81e03489837416d84b2b7a24728a933, role_id=4207a743edc04b0aba9197fe3554b2e3)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:16.405705 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfac7d8-a4a1-47d1-b044-c3613bc6df2d None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:16.425270 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 121/234] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Aug 23 04:14:16 2022] PUT /identity/v3/projects/95a15538272e426ba99c0b50e2f30f52/users/d81e03489837416d84b2b7a24728a933/roles/4207a743edc04b0aba9197fe3554b2e3 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:14:16.834697 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb5548-2c15-482d-9ae4-af3fee31d8dc None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:16.835188 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb5548-2c15-482d-9ae4-af3fee31d8dc None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:16.835188 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb5548-2c15-482d-9ae4-af3fee31d8dc None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:16.837035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 114/235] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:16.845407 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23364eaf-0475-4ddc-9f39-a0703586465b None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:16.845854 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23364eaf-0475-4ddc-9f39-a0703586465b None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:16.846033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23364eaf-0475-4ddc-9f39-a0703586465b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:16.872125 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-23364eaf-0475-4ddc-9f39-a0703586465b None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:16.873341 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-23364eaf-0475-4ddc-9f39-a0703586465b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:16.926067 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d16af5-2916-4315-b8a6-01c87b6f596e None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:16.927598 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d16af5-2916-4315-b8a6-01c87b6f596e None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:16.928113 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d16af5-2916-4315-b8a6-01c87b6f596e None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:16.930011 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 115/236] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:16.938367 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc645a8-023a-4d7f-8035-d5cca024e126 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:16.939389 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc645a8-023a-4d7f-8035-d5cca024e126 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:16.940182 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc645a8-023a-4d7f-8035-d5cca024e126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:16.977165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-1bc645a8-023a-4d7f-8035-d5cca024e126 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:16.978383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-1bc645a8-023a-4d7f-8035-d5cca024e126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:16.984501 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 122/237] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 04:14:17.087270 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 116/238] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:17.260338 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23364eaf-0475-4ddc-9f39-a0703586465b None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:17.354235 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:17.363685 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23364eaf-0475-4ddc-9f39-a0703586465b None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:17.400682 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314636a-502c-43cb-91db-7ac8df0e280e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:17.401326 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314636a-502c-43cb-91db-7ac8df0e280e None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:17.401597 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314636a-502c-43cb-91db-7ac8df0e280e None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:17.401861 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314636a-502c-43cb-91db-7ac8df0e280e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:17.403047 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2314636a-502c-43cb-91db-7ac8df0e280e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:17.404551 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2314636a-502c-43cb-91db-7ac8df0e280e None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:17.406456 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 123/239] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:14:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:17.411992 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc645a8-023a-4d7f-8035-d5cca024e126 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:17.516696 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:17.531743 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc645a8-023a-4d7f-8035-d5cca024e126 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:17.557050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314636a-502c-43cb-91db-7ac8df0e280e None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:17.568110 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e505c-4e4e-44ca-a444-161ea2718016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:17.568730 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e505c-4e4e-44ca-a444-161ea2718016 None admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:17.569003 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e505c-4e4e-44ca-a444-161ea2718016 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:17.569278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e505c-4e4e-44ca-a444-161ea2718016 None admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:17.569940 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e505c-4e4e-44ca-a444-161ea2718016 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:17.571459 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e505c-4e4e-44ca-a444-161ea2718016 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:17.602777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 117/240] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Aug 23 04:14:17 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:14:17.692395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314636a-502c-43cb-91db-7ac8df0e280e None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:17.735474 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e8e10-51cb-447c-8234-a41ffef4f42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:17.736160 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e8e10-51cb-447c-8234-a41ffef4f42c None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:17.736440 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e8e10-51cb-447c-8234-a41ffef4f42c None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:17.736714 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e8e10-51cb-447c-8234-a41ffef4f42c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:17.749718 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4e8e10-51cb-447c-8234-a41ffef4f42c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:17.754758 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-4c4e8e10-51cb-447c-8234-a41ffef4f42c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 04:14:17.754758 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:17.754758 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:17.754758 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:17.754758 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:17.754758 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:17.754758 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:17.754758 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:17.754758 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:17.754758 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:17.754758 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:17.754758 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:17.754758 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:17.754758 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:17.754758 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:17.754758 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:17.755700 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:17.755700 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:17.755700 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:17.755700 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:17.755700 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:17.755700 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:17.755700 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:17.755700 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:17.755700 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:17.755700 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:17.755700 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:17.755700 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:17.755700 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:17.755700 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:17.755700 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:17.755700 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:17.756554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:17.756554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:17.756554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:17.756554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:17.756554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:17.756554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:17.756554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:17.756554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:17.756554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:17.756554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:17.756554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 04:14:17.756554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:17.757684 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 124/241] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:17.762989 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e505c-4e4e-44ca-a444-161ea2718016 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:17.897732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e505c-4e4e-44ca-a444-161ea2718016 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:17.938534 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efdb7f2-9c8c-4ce5-a548-8eefa0ef3e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:17.939560 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efdb7f2-9c8c-4ce5-a548-8eefa0ef3e38 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:17.940077 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efdb7f2-9c8c-4ce5-a548-8eefa0ef3e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:17.940573 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efdb7f2-9c8c-4ce5-a548-8eefa0ef3e38 None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:17.941445 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efdb7f2-9c8c-4ce5-a548-8eefa0ef3e38 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:17.941943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efdb7f2-9c8c-4ce5-a548-8eefa0ef3e38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:17.943570 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efdb7f2-9c8c-4ce5-a548-8eefa0ef3e38 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:17.953051 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 118/242] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Aug 23 04:14:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:17.958410 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e8e10-51cb-447c-8234-a41ffef4f42c None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:17.981432 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:18.027620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb653248-ad62-4c9a-9873-3730a1fed0d9 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:18.027906 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb653248-ad62-4c9a-9873-3730a1fed0d9 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:18.028189 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb653248-ad62-4c9a-9873-3730a1fed0d9 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:18.029673 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 119/243] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:14:18.035318 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd7168-7e70-430f-bfd4-60894bf49328 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:18.035609 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd7168-7e70-430f-bfd4-60894bf49328 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:18.036157 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd7168-7e70-430f-bfd4-60894bf49328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:18.047044 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:18.061856 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-9dcd7168-7e70-430f-bfd4-60894bf49328 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:18.063183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-9dcd7168-7e70-430f-bfd4-60894bf49328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:18.096758 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e8e10-51cb-447c-8234-a41ffef4f42c None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:18.147044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431cc024-6e19-4d14-9eed-88aab262295a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:18.147044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cc024-6e19-4d14-9eed-88aab262295a None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:18.147044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cc024-6e19-4d14-9eed-88aab262295a None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:18.147044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cc024-6e19-4d14-9eed-88aab262295a None admin] PATH_INFO: `/v3/users/98f73a46b3254849be8c8d175b2ac40c` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:18.147811 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 120/244] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 04:14:18.170747 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431cc024-6e19-4d14-9eed-88aab262295a None admin] RBAC: Authorizing `identity:get_user(user_id=98f73a46b3254849be8c8d175b2ac40c)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:18.172288 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431cc024-6e19-4d14-9eed-88aab262295a None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:18.174016 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 125/245] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:14:17 2022] GET /identity/v3/users/98f73a46b3254849be8c8d175b2ac40c => generated 297 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:14:18.179928 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd7168-7e70-430f-bfd4-60894bf49328 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:18.291002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd7168-7e70-430f-bfd4-60894bf49328 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:18.326170 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db476f52-d7ce-4246-bee6-105c3674fdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:18.326796 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db476f52-d7ce-4246-bee6-105c3674fdbe None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:18.327074 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db476f52-d7ce-4246-bee6-105c3674fdbe None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:18.327345 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db476f52-d7ce-4246-bee6-105c3674fdbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:18.332216 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:18.401305 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db476f52-d7ce-4246-bee6-105c3674fdbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:18.402771 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db476f52-d7ce-4246-bee6-105c3674fdbe None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:18.466567 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431cc024-6e19-4d14-9eed-88aab262295a None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:18.512099 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 121/246] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:14:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:14:18.516197 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db476f52-d7ce-4246-bee6-105c3674fdbe None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:18.560426 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:18.568187 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:18.611437 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431cc024-6e19-4d14-9eed-88aab262295a None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:18.678367 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db476f52-d7ce-4246-bee6-105c3674fdbe None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:18.686101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f0078-be6a-42bf-914e-61a9f84e06d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:18.686749 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f0078-be6a-42bf-914e-61a9f84e06d9 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:18.687022 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f0078-be6a-42bf-914e-61a9f84e06d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:18.687285 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f0078-be6a-42bf-914e-61a9f84e06d9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:18.709513 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8f0078-be6a-42bf-914e-61a9f84e06d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:18.715172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-1e8f0078-be6a-42bf-914e-61a9f84e06d9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:18.715172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:18.715172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:18.715172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:18.715172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:18.715172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:18.715172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:18.715172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:18.715172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:18.715172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:18.715172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:18.715172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:18.715172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:18.715172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:18.715172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:18.715172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:18.716151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:18.716151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:18.716151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:18.716151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:18.716151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:18.716151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:18.716151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:18.716151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:18.716151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:18.716151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:18.716151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:18.716151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:18.716151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:18.716151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:18.716151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:18.716151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:18.716936 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:18.716936 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:18.716936 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:18.716936 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:18.716936 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:18.716936 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:18.716936 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:18.716936 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:18.716936 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:18.716936 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:18.716936 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:18.716936 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:18.716936 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 126/247] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:18.717564 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f22c0-3a90-4267-b06e-b197abae8ae8 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:18.722989 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f22c0-3a90-4267-b06e-b197abae8ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:18.722989 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f22c0-3a90-4267-b06e-b197abae8ae8 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:18.722989 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 127/248] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:18.727157 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f22c0-3a90-4267-b06e-b197abae8ae8 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:18.747895 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d01d3e-070c-42d6-a118-66dbdb20c636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:18.749322 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d01d3e-070c-42d6-a118-66dbdb20c636 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:18.750013 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d01d3e-070c-42d6-a118-66dbdb20c636 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:18.751577 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d01d3e-070c-42d6-a118-66dbdb20c636 None admin] PATH_INFO: `/v3/projects/95a15538272e426ba99c0b50e2f30f52` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:18.754153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d01d3e-070c-42d6-a118-66dbdb20c636 None admin] RBAC: Authorizing `identity:get_project(project_id=95a15538272e426ba99c0b50e2f30f52)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:18.755815 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d01d3e-070c-42d6-a118-66dbdb20c636 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:18.759322 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 122/249] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:18 2022] GET /identity/v3/projects/95a15538272e426ba99c0b50e2f30f52 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:14:18.761649 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b8ad5-5ed5-4563-8c9f-86edded652df None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:18.762931 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b8ad5-5ed5-4563-8c9f-86edded652df None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:18.764968 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b8ad5-5ed5-4563-8c9f-86edded652df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:18.799576 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-e59b8ad5-5ed5-4563-8c9f-86edded652df None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:18.801584 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-e59b8ad5-5ed5-4563-8c9f-86edded652df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:18.903337 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f22c0-3a90-4267-b06e-b197abae8ae8 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:18.903683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 123/250] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:18.905270 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b8ad5-5ed5-4563-8c9f-86edded652df None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:18.948302 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:18.965374 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb9084-119d-4a23-9f27-1f24dde3a67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:18.966827 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb9084-119d-4a23-9f27-1f24dde3a67f None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:18.967297 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb9084-119d-4a23-9f27-1f24dde3a67f None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:18.967778 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb9084-119d-4a23-9f27-1f24dde3a67f None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:18.968671 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeb9084-119d-4a23-9f27-1f24dde3a67f None admin] RBAC: Adding query filter params (name=member) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:18.969164 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeb9084-119d-4a23-9f27-1f24dde3a67f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:18.970830 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeb9084-119d-4a23-9f27-1f24dde3a67f None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:18.976483 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:18.997634 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 128/251] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Aug 23 04:14:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:19.003646 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c7080-9bed-4559-9ba2-93a8af17c11c None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:19.004271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c7080-9bed-4559-9ba2-93a8af17c11c None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:19.007088 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c7080-9bed-4559-9ba2-93a8af17c11c None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:19.008941 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 129/252] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:18 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:19.014184 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c7080-9bed-4559-9ba2-93a8af17c11c None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:19.119230 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b8ad5-5ed5-4563-8c9f-86edded652df None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:19.189558 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4eebb-d4fe-45e7-bd9b-d0cd2a4f1424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:19.190247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4eebb-d4fe-45e7-bd9b-d0cd2a4f1424 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:19.190643 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4eebb-d4fe-45e7-bd9b-d0cd2a4f1424 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:19.190923 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4eebb-d4fe-45e7-bd9b-d0cd2a4f1424 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:19.191674 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e4eebb-d4fe-45e7-bd9b-d0cd2a4f1424 None admin] RBAC: Adding query filter params (role.id=4be260163ba342e9b42232a49eec9fec, scope.project.id=95a15538272e426ba99c0b50e2f30f52, user.id=98f73a46b3254849be8c8d175b2ac40c) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:19.192017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e4eebb-d4fe-45e7-bd9b-d0cd2a4f1424 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:19.193477 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e4eebb-d4fe-45e7-bd9b-d0cd2a4f1424 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:19.194190 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c7080-9bed-4559-9ba2-93a8af17c11c None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:19.200285 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 124/253] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:14:18 2022] GET /identity/v3/role_assignments?user.id=98f73a46b3254849be8c8d175b2ac40c&scope.project.id=95a15538272e426ba99c0b50e2f30f52&role.id=4be260163ba342e9b42232a49eec9fec => generated 259 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:19.201474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3ac17-de4f-4690-afbe-951286ca3754 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:19.201758 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3ac17-de4f-4690-afbe-951286ca3754 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:19.202032 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3ac17-de4f-4690-afbe-951286ca3754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:19.228471 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-27f3ac17-de4f-4690-afbe-951286ca3754 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:19.229731 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-27f3ac17-de4f-4690-afbe-951286ca3754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:19.279673 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f7437-b902-4300-9746-42ebf5d35954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:19.280370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f7437-b902-4300-9746-42ebf5d35954 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:19.280649 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f7437-b902-4300-9746-42ebf5d35954 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:19.280920 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f7437-b902-4300-9746-42ebf5d35954 None admin] PATH_INFO: `/v3/users/4772005528a1418a904ae1d87d9e0f18` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:19.283523 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f7437-b902-4300-9746-42ebf5d35954 None admin] RBAC: Authorizing `identity:get_user(user_id=4772005528a1418a904ae1d87d9e0f18)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:19.285048 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f7437-b902-4300-9746-42ebf5d35954 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:19.286776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 130/254] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:14:19 2022] GET /identity/v3/users/4772005528a1418a904ae1d87d9e0f18 => generated 297 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:19.287550 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f7437-b902-4300-9746-42ebf5d35954 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:19.379004 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:19.386654 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 125/255] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:19.396337 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f3ac17-de4f-4690-afbe-951286ca3754 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:19.418563 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f7437-b902-4300-9746-42ebf5d35954 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:19.452378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:19.472331 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e080511-f599-46ec-88d3-8221d67b3e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:19.473231 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e080511-f599-46ec-88d3-8221d67b3e36 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:19.473654 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e080511-f599-46ec-88d3-8221d67b3e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:19.474069 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e080511-f599-46ec-88d3-8221d67b3e36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:19.486614 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e080511-f599-46ec-88d3-8221d67b3e36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:19.492594 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-4e080511-f599-46ec-88d3-8221d67b3e36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:14:19.492594 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:19.492594 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:19.492594 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:19.492594 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:19.492594 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:19.492594 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:19.492594 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:19.492594 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:19.492594 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:19.492594 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:19.492594 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:19.492594 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:19.492594 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:19.492594 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:19.492594 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:19.493632 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:19.493632 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:19.493632 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:19.493632 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:19.493632 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:19.493632 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:19.493632 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:19.493632 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:19.493632 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:19.493632 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:19.493632 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:19.493632 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:19.493632 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:19.493632 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:19.493632 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:19.493632 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:19.494567 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:19.494567 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:19.494567 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:19.494567 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:19.494567 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:19.494567 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:19.494567 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:19.494567 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:19.494567 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:19.494567 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:19.494567 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:14:19.494567 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:19.496965 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 131/256] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:14:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:19.502841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e080511-f599-46ec-88d3-8221d67b3e36 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:19.633657 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f3ac17-de4f-4690-afbe-951286ca3754 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:19.650098 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e080511-f599-46ec-88d3-8221d67b3e36 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:19.717027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:19.728901 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1633c-5aea-4a99-89c0-4baf11c6e1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:19.729544 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1633c-5aea-4a99-89c0-4baf11c6e1db None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:19.729819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1633c-5aea-4a99-89c0-4baf11c6e1db None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:19.730086 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1633c-5aea-4a99-89c0-4baf11c6e1db None admin] PATH_INFO: `/v3/projects/95a15538272e426ba99c0b50e2f30f52` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:19.731604 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c1633c-5aea-4a99-89c0-4baf11c6e1db None admin] RBAC: Authorizing `identity:get_project(project_id=95a15538272e426ba99c0b50e2f30f52)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:19.734835 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c1633c-5aea-4a99-89c0-4baf11c6e1db None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:19.736504 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 126/257] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:19 2022] GET /identity/v3/projects/95a15538272e426ba99c0b50e2f30f52 => generated 302 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:19.742465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c43d4b-fa84-4670-99a0-6c91b0267684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:19.743086 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c43d4b-fa84-4670-99a0-6c91b0267684 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:19.743365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c43d4b-fa84-4670-99a0-6c91b0267684 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:19.743632 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c43d4b-fa84-4670-99a0-6c91b0267684 None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:19.744338 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c43d4b-fa84-4670-99a0-6c91b0267684 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:19.744639 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c43d4b-fa84-4670-99a0-6c91b0267684 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:19.746031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c43d4b-fa84-4670-99a0-6c91b0267684 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:19.748075 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1633c-5aea-4a99-89c0-4baf11c6e1db None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:19.767350 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 132/258] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:14:19.774429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c43d4b-fa84-4670-99a0-6c91b0267684 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:19.934556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c43d4b-fa84-4670-99a0-6c91b0267684 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:19.977538 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e180b-8fdd-437b-a629-7ce21d1b3ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:19.978153 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e180b-8fdd-437b-a629-7ce21d1b3ecc None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:19.978378 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e180b-8fdd-437b-a629-7ce21d1b3ecc None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:19.978595 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e180b-8fdd-437b-a629-7ce21d1b3ecc None admin] PATH_INFO: `/v3/users/d81e03489837416d84b2b7a24728a933` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:19.980547 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e180b-8fdd-437b-a629-7ce21d1b3ecc None admin] RBAC: Authorizing `identity:get_user(user_id=d81e03489837416d84b2b7a24728a933)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:19.982017 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e180b-8fdd-437b-a629-7ce21d1b3ecc None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:19.983626 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 133/259] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:14:19 2022] GET /identity/v3/users/d81e03489837416d84b2b7a24728a933 => generated 283 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:14:19.986312 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:19.989334 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e180b-8fdd-437b-a629-7ce21d1b3ecc None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:19.998459 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1633c-5aea-4a99-89c0-4baf11c6e1db None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:20.063054 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa3aac-a41e-42af-9c67-1bd2c5827a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:20.063719 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa3aac-a41e-42af-9c67-1bd2c5827a51 None admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:20.064013 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa3aac-a41e-42af-9c67-1bd2c5827a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:20.064285 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa3aac-a41e-42af-9c67-1bd2c5827a51 None admin] PATH_INFO: `/v3/projects/95a15538272e426ba99c0b50e2f30f52/users/4772005528a1418a904ae1d87d9e0f18/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:20.069178 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa3aac-a41e-42af-9c67-1bd2c5827a51 None admin] RBAC: Authorizing `identity:create_grant(project_id=95a15538272e426ba99c0b50e2f30f52, user_id=4772005528a1418a904ae1d87d9e0f18, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:20.070967 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa3aac-a41e-42af-9c67-1bd2c5827a51 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:20.094975 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 127/260] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Aug 23 04:14:19 2022] PUT /identity/v3/projects/95a15538272e426ba99c0b50e2f30f52/users/4772005528a1418a904ae1d87d9e0f18/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:14:20.097116 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa3aac-a41e-42af-9c67-1bd2c5827a51 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:20.102667 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:20.181210 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e180b-8fdd-437b-a629-7ce21d1b3ecc None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:20.235755 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8488e21-509d-450e-88d8-35aa8617b573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:20.236350 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8488e21-509d-450e-88d8-35aa8617b573 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:20.236649 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8488e21-509d-450e-88d8-35aa8617b573 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:20.237246 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8488e21-509d-450e-88d8-35aa8617b573 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:20.252129 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8488e21-509d-450e-88d8-35aa8617b573 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:20.259033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-a8488e21-509d-450e-88d8-35aa8617b573 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 04:14:20.259033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:20.259033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:20.259033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:20.259033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:20.259033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:20.259033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:20.259033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:20.259033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:20.259033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:20.259033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:20.259033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:20.259033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:20.259033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:20.259033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:20.259033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:20.260013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:20.260013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:20.260013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:20.260013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:20.260013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:20.260013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:20.260013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:20.260013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:20.260013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:20.260013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:20.260013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:20.260013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:20.260013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:20.260013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:20.260013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:20.260013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:20.260804 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:20.260804 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:20.260804 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:20.260804 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:20.260804 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:20.260804 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:20.260804 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:20.260804 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:20.260804 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:20.260804 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:20.260804 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 04:14:20.260804 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:20.264744 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 134/261] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Aug 23 04:14:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:20.267380 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8488e21-509d-450e-88d8-35aa8617b573 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:20.302855 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa3aac-a41e-42af-9c67-1bd2c5827a51 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:20.357549 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75120791-5e0c-42ac-9ad0-523e0cbc4521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:20.358174 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75120791-5e0c-42ac-9ad0-523e0cbc4521 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:20.358446 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75120791-5e0c-42ac-9ad0-523e0cbc4521 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:20.358714 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75120791-5e0c-42ac-9ad0-523e0cbc4521 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:20.386350 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:20.420005 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8488e21-509d-450e-88d8-35aa8617b573 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:20.440077 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75120791-5e0c-42ac-9ad0-523e0cbc4521 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:20.441572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75120791-5e0c-42ac-9ad0-523e0cbc4521 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:20.457636 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:20.485528 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780dbbc6-455e-49fd-a1e6-042514d2038f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:20.486208 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780dbbc6-455e-49fd-a1e6-042514d2038f None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:20.486435 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780dbbc6-455e-49fd-a1e6-042514d2038f None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:20.486712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780dbbc6-455e-49fd-a1e6-042514d2038f None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:20.487396 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780dbbc6-455e-49fd-a1e6-042514d2038f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:20.487689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780dbbc6-455e-49fd-a1e6-042514d2038f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:20.489110 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780dbbc6-455e-49fd-a1e6-042514d2038f None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:20.502138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 135/262] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:14:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:14:20.508390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780dbbc6-455e-49fd-a1e6-042514d2038f None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:20.557743 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 128/263] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:14:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:14:20.563351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75120791-5e0c-42ac-9ad0-523e0cbc4521 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:20.672275 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:20.697924 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780dbbc6-455e-49fd-a1e6-042514d2038f None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:20.723750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75120791-5e0c-42ac-9ad0-523e0cbc4521 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:20.766495 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a94bbb-d5af-47f4-b49a-dd7f42cf3eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:20.767058 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a94bbb-d5af-47f4-b49a-dd7f42cf3eab None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:20.767331 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a94bbb-d5af-47f4-b49a-dd7f42cf3eab None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:20.767599 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a94bbb-d5af-47f4-b49a-dd7f42cf3eab None admin] PATH_INFO: `/v3/projects/95a15538272e426ba99c0b50e2f30f52` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:20.768779 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a94bbb-d5af-47f4-b49a-dd7f42cf3eab None admin] RBAC: Authorizing `identity:get_project(project_id=95a15538272e426ba99c0b50e2f30f52)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:20.770587 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a94bbb-d5af-47f4-b49a-dd7f42cf3eab None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:20.772333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9e518-9101-4af4-8083-aa839a5415fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:20.772471 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 129/264] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:20 2022] GET /identity/v3/projects/95a15538272e426ba99c0b50e2f30f52 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:14:20.773315 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9e518-9101-4af4-8083-aa839a5415fa None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:20.773424 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9e518-9101-4af4-8083-aa839a5415fa None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:20.773768 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9e518-9101-4af4-8083-aa839a5415fa None admin] PATH_INFO: `/v3/users/d81e03489837416d84b2b7a24728a933` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:20.777459 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9e518-9101-4af4-8083-aa839a5415fa None admin] RBAC: Authorizing `identity:get_user(user_id=d81e03489837416d84b2b7a24728a933)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:20.778206 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648e590-e80d-4244-b42c-b19f50d7f051 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:20.778483 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648e590-e80d-4244-b42c-b19f50d7f051 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:20.778751 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648e590-e80d-4244-b42c-b19f50d7f051 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:20.780339 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 130/265] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:20.781395 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426af09-37af-45b5-978c-b692a5a34aa3 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:20.781652 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9e518-9101-4af4-8083-aa839a5415fa None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:20.781751 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426af09-37af-45b5-978c-b692a5a34aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:20.781983 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426af09-37af-45b5-978c-b692a5a34aa3 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:20.783405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 131/266] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:20.784203 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 136/267] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:14:20 2022] GET /identity/v3/users/d81e03489837416d84b2b7a24728a933 => generated 283 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:20.786395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d75985-09de-4358-9e2e-0d660c5aef2a None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:20.786887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d75985-09de-4358-9e2e-0d660c5aef2a None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:20.787353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d75985-09de-4358-9e2e-0d660c5aef2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:20.791937 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426af09-37af-45b5-978c-b692a5a34aa3 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:20.810638 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-56d75985-09de-4358-9e2e-0d660c5aef2a None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:20.811850 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-56d75985-09de-4358-9e2e-0d660c5aef2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:20.857261 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:20.882651 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 137/268] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:20.883780 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2a79a-50ad-439c-bf46-3002e3f0f3a3 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:20.884060 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2a79a-50ad-439c-bf46-3002e3f0f3a3 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:20.884339 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2a79a-50ad-439c-bf46-3002e3f0f3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:20.902208 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-40a2a79a-50ad-439c-bf46-3002e3f0f3a3 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:20.902961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426af09-37af-45b5-978c-b692a5a34aa3 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:20.903446 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-40a2a79a-50ad-439c-bf46-3002e3f0f3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:20.948273 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:20.969432 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e8893d-4de9-47bd-9cd3-3ff871c92557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:20.970053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8893d-4de9-47bd-9cd3-3ff871c92557 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:20.970333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8893d-4de9-47bd-9cd3-3ff871c92557 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:20.970599 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8893d-4de9-47bd-9cd3-3ff871c92557 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:20.971342 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e8893d-4de9-47bd-9cd3-3ff871c92557 None admin] RBAC: Adding query filter params (role.id=52b31061b72c45ccb76dd26bf31f8d3f, scope.project.id=95a15538272e426ba99c0b50e2f30f52, user.id=d81e03489837416d84b2b7a24728a933) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:20.971649 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e8893d-4de9-47bd-9cd3-3ff871c92557 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:20.973130 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e8893d-4de9-47bd-9cd3-3ff871c92557 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:20.991481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 138/269] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:20.993019 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2a79a-50ad-439c-bf46-3002e3f0f3a3 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:20.993395 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 132/270] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:14:20 2022] GET /identity/v3/role_assignments?user.id=d81e03489837416d84b2b7a24728a933&scope.project.id=95a15538272e426ba99c0b50e2f30f52&role.id=52b31061b72c45ccb76dd26bf31f8d3f => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:21.106342 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2a79a-50ad-439c-bf46-3002e3f0f3a3 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:21.115030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:21.150083 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be592d-bcb7-4b71-b890-eae6289b74e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:21.151094 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be592d-bcb7-4b71-b890-eae6289b74e1 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:21.151568 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be592d-bcb7-4b71-b890-eae6289b74e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:21.152082 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be592d-bcb7-4b71-b890-eae6289b74e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:21.223836 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be592d-bcb7-4b71-b890-eae6289b74e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:21.225718 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be592d-bcb7-4b71-b890-eae6289b74e1 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:21.328470 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 139/271] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:14:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:14:21.335736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e8893d-4de9-47bd-9cd3-3ff871c92557 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:21.351826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be592d-bcb7-4b71-b890-eae6289b74e1 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:21.363329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:21.505538 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e8893d-4de9-47bd-9cd3-3ff871c92557 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:21.532999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be592d-bcb7-4b71-b890-eae6289b74e1 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:21.549382 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78264f5a-9a73-4a1e-bd8a-118ad36b2c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:21.550600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78264f5a-9a73-4a1e-bd8a-118ad36b2c66 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:21.551500 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78264f5a-9a73-4a1e-bd8a-118ad36b2c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:21.552238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78264f5a-9a73-4a1e-bd8a-118ad36b2c66 None admin] PATH_INFO: `/v3/projects/95a15538272e426ba99c0b50e2f30f52` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:21.553643 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78264f5a-9a73-4a1e-bd8a-118ad36b2c66 None admin] RBAC: Authorizing `identity:get_project(project_id=95a15538272e426ba99c0b50e2f30f52)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:21.555935 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78264f5a-9a73-4a1e-bd8a-118ad36b2c66 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:21.562386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 133/272] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:21 2022] GET /identity/v3/projects/95a15538272e426ba99c0b50e2f30f52 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:21.562386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78264f5a-9a73-4a1e-bd8a-118ad36b2c66 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:21.585476 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a0eec-c93d-47a6-b029-7dc9693b3513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:21.586901 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a0eec-c93d-47a6-b029-7dc9693b3513 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:21.587374 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a0eec-c93d-47a6-b029-7dc9693b3513 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:21.587829 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a0eec-c93d-47a6-b029-7dc9693b3513 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:21.589579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19a0eec-c93d-47a6-b029-7dc9693b3513 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:21.591812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19a0eec-c93d-47a6-b029-7dc9693b3513 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:21.595010 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 140/273] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:14:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:14:21.596468 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a0eec-c93d-47a6-b029-7dc9693b3513 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:21.628329 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:21.682317 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78264f5a-9a73-4a1e-bd8a-118ad36b2c66 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:21.696082 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:21.723612 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3815e4a6-0174-4415-a251-1cc100167c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:21.725108 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815e4a6-0174-4415-a251-1cc100167c56 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:21.725528 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815e4a6-0174-4415-a251-1cc100167c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:21.725934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815e4a6-0174-4415-a251-1cc100167c56 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:21.727381 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a0eec-c93d-47a6-b029-7dc9693b3513 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:21.738984 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3815e4a6-0174-4415-a251-1cc100167c56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:21.744455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-3815e4a6-0174-4415-a251-1cc100167c56 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:21.744455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:21.744455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:21.744455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:21.744455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:21.744455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:21.744455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:21.744455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:21.744455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:21.744455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:21.744455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:21.744455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:21.744455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:21.744455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:21.744455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:21.744455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:21.745317 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:21.745317 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:21.745317 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:21.745317 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:21.745317 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:21.745317 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:21.745317 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:21.745317 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:21.745317 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:21.745317 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:21.745317 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:21.745317 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:21.745317 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:21.745317 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:21.745317 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:21.745317 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:21.746121 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:21.746121 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:21.746121 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:21.746121 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:21.746121 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:21.746121 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:21.746121 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:21.746121 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:21.746121 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:21.746121 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:21.746121 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:21.746121 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:21.746121 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 134/274] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:21.747505 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3815e4a6-0174-4415-a251-1cc100167c56 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:21.768988 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63041c3-ffd7-487b-b1f3-85aff64c19bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:21.769600 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63041c3-ffd7-487b-b1f3-85aff64c19bc None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:21.769880 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63041c3-ffd7-487b-b1f3-85aff64c19bc None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:21.770154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63041c3-ffd7-487b-b1f3-85aff64c19bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:21.770898 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63041c3-ffd7-487b-b1f3-85aff64c19bc None admin] RBAC: Adding query filter params (role.id=4207a743edc04b0aba9197fe3554b2e3, scope.project.id=95a15538272e426ba99c0b50e2f30f52, user.id=d81e03489837416d84b2b7a24728a933) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:21.771657 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63041c3-ffd7-487b-b1f3-85aff64c19bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:21.773801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63041c3-ffd7-487b-b1f3-85aff64c19bc None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:21.788343 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 141/275] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:14:21 2022] GET /identity/v3/role_assignments?user.id=d81e03489837416d84b2b7a24728a933&scope.project.id=95a15538272e426ba99c0b50e2f30f52&role.id=4207a743edc04b0aba9197fe3554b2e3 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:21.788343 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422de8c-ff54-42d0-83e6-fe06851896cb None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:21.788343 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422de8c-ff54-42d0-83e6-fe06851896cb None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:21.788343 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422de8c-ff54-42d0-83e6-fe06851896cb None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:21.788343 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 142/276] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:21.788343 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422de8c-ff54-42d0-83e6-fe06851896cb None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:21.869299 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3815e4a6-0174-4415-a251-1cc100167c56 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:21.890654 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:21.920599 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e129c2-3307-42a2-b2e7-aa8ae0263e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:21.921233 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e129c2-3307-42a2-b2e7-aa8ae0263e16 None admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:21.921515 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e129c2-3307-42a2-b2e7-aa8ae0263e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:21.921679 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422de8c-ff54-42d0-83e6-fe06851896cb None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:21.921789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e129c2-3307-42a2-b2e7-aa8ae0263e16 None admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:21.922446 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e129c2-3307-42a2-b2e7-aa8ae0263e16 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:21.925258 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e129c2-3307-42a2-b2e7-aa8ae0263e16 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:21.957969 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 135/277] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Aug 23 04:14:21 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:14:21.959500 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408a2c4-f2c8-451e-97ed-4f846efa80fb None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:21.959953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408a2c4-f2c8-451e-97ed-4f846efa80fb None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:21.960238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408a2c4-f2c8-451e-97ed-4f846efa80fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:21.977159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65fb703-c48f-4093-93ef-5779220745b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:21.977549 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fb703-c48f-4093-93ef-5779220745b9 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:21.977782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fb703-c48f-4093-93ef-5779220745b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:21.978059 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fb703-c48f-4093-93ef-5779220745b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:21.978753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65fb703-c48f-4093-93ef-5779220745b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:21.979061 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65fb703-c48f-4093-93ef-5779220745b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:21.980575 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65fb703-c48f-4093-93ef-5779220745b9 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:21.987757 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:21.988115 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-f408a2c4-f2c8-451e-97ed-4f846efa80fb None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:21.989482 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-f408a2c4-f2c8-451e-97ed-4f846efa80fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:21.993001 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 143/278] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Aug 23 04:14:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:22.003744 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65fb703-c48f-4093-93ef-5779220745b9 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:22.115205 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 136/279] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:22.146655 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65fb703-c48f-4093-93ef-5779220745b9 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:22.183579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4467eb1-f125-4ec3-b4e8-c60a70e9c3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:22.184318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4467eb1-f125-4ec3-b4e8-c60a70e9c3bc None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:22.184638 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4467eb1-f125-4ec3-b4e8-c60a70e9c3bc None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:22.184912 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4467eb1-f125-4ec3-b4e8-c60a70e9c3bc None admin] PATH_INFO: `/v3/users/78e7881aab9b46fda160b9662233c76e` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:22.199334 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4467eb1-f125-4ec3-b4e8-c60a70e9c3bc None admin] RBAC: Authorizing `identity:get_user(user_id=78e7881aab9b46fda160b9662233c76e)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:22.200926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4467eb1-f125-4ec3-b4e8-c60a70e9c3bc None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:22.203072 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 144/280] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:14:21 2022] GET /identity/v3/users/78e7881aab9b46fda160b9662233c76e => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:22.208684 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408a2c4-f2c8-451e-97ed-4f846efa80fb None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:22.319479 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:22.323156 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408a2c4-f2c8-451e-97ed-4f846efa80fb None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:22.365839 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d465ac-17df-47bd-ae8e-992bb8493f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:22.366463 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d465ac-17df-47bd-ae8e-992bb8493f04 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:22.366871 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d465ac-17df-47bd-ae8e-992bb8493f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:22.367632 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d465ac-17df-47bd-ae8e-992bb8493f04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:22.368406 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d465ac-17df-47bd-ae8e-992bb8493f04 None admin] RBAC: Adding query filter params (role.id=aa03504b4fb642d288d07f4335e291e1, user.id=78e7881aab9b46fda160b9662233c76e, scope.system=all) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:22.368756 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d465ac-17df-47bd-ae8e-992bb8493f04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:22.370163 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d465ac-17df-47bd-ae8e-992bb8493f04 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:22.377494 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 137/281] 10.222.0.18 () {56 vars in 1308 bytes} [Tue Aug 23 04:14:22 2022] GET /identity/v3/role_assignments?user.id=78e7881aab9b46fda160b9662233c76e&scope.system=all&role.id=aa03504b4fb642d288d07f4335e291e1 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:14:22.397457 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4467eb1-f125-4ec3-b4e8-c60a70e9c3bc None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:22.474655 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:22.524296 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4467eb1-f125-4ec3-b4e8-c60a70e9c3bc None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:22.578209 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d584f4-7773-453f-a5b1-5ba19581bcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:22.578874 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d584f4-7773-453f-a5b1-5ba19581bcb6 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:22.579146 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d584f4-7773-453f-a5b1-5ba19581bcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:22.579413 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d584f4-7773-453f-a5b1-5ba19581bcb6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:22.593815 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d584f4-7773-453f-a5b1-5ba19581bcb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:22.599329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-a1d584f4-7773-453f-a5b1-5ba19581bcb6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 04:14:22.599329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:22.599329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:22.599329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:22.599329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:22.599329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:22.599329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:22.599329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:22.599329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:22.599329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:22.599329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:22.599329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:22.599329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:22.599329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:22.599329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:22.599329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:22.600237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:22.600237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:22.600237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:22.600237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:22.600237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:22.600237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:22.600237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:22.600237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:22.600237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:22.600237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:22.600237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:22.600237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:22.600237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:22.600237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:22.600237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:22.600237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:22.601033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:22.601033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:22.601033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:22.601033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:22.601033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:22.601033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:22.601033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:22.601033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:22.601033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:22.601033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:22.601033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 04:14:22.601033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:22.601033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 145/282] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:14:22.606670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d465ac-17df-47bd-ae8e-992bb8493f04 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:22.693687 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09c44c-a98a-4b60-adfa-0464fdec7618 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:22.694307 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09c44c-a98a-4b60-adfa-0464fdec7618 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:22.694837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09c44c-a98a-4b60-adfa-0464fdec7618 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:22.696928 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 146/283] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:22.704348 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58d4db-d112-4515-9d83-cd6ce04d96b6 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:22.704991 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58d4db-d112-4515-9d83-cd6ce04d96b6 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:22.705461 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58d4db-d112-4515-9d83-cd6ce04d96b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:22.743677 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d465ac-17df-47bd-ae8e-992bb8493f04 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:22.748964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-0b58d4db-d112-4515-9d83-cd6ce04d96b6 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:22.750177 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-0b58d4db-d112-4515-9d83-cd6ce04d96b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:22.778885 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f35ec-6a84-4764-a95b-5edfddc8324d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:22.779503 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f35ec-6a84-4764-a95b-5edfddc8324d None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:22.779781 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f35ec-6a84-4764-a95b-5edfddc8324d None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:22.780175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f35ec-6a84-4764-a95b-5edfddc8324d None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:22.780947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567f35ec-6a84-4764-a95b-5edfddc8324d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:22.781252 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567f35ec-6a84-4764-a95b-5edfddc8324d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:22.782643 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567f35ec-6a84-4764-a95b-5edfddc8324d None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:22.796063 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 138/284] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Aug 23 04:14:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:22.797720 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f35ec-6a84-4764-a95b-5edfddc8324d None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:22.838092 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:22.885308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 147/285] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:22.918018 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f35ec-6a84-4764-a95b-5edfddc8324d None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:22.953685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f60dc9-481d-46c9-956b-bf7165a4c42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:22.954242 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f60dc9-481d-46c9-956b-bf7165a4c42c None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:22.954520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f60dc9-481d-46c9-956b-bf7165a4c42c None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:22.954790 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f60dc9-481d-46c9-956b-bf7165a4c42c None admin] PATH_INFO: `/v3/users/98f73a46b3254849be8c8d175b2ac40c` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:22.955940 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f60dc9-481d-46c9-956b-bf7165a4c42c None admin] RBAC: Authorizing `identity:get_user(user_id=98f73a46b3254849be8c8d175b2ac40c)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:22.957641 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f60dc9-481d-46c9-956b-bf7165a4c42c None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:22.962998 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 139/286] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:14:22 2022] GET /identity/v3/users/98f73a46b3254849be8c8d175b2ac40c => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:22.969026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58d4db-d112-4515-9d83-cd6ce04d96b6 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:23.043943 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:23.111321 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58d4db-d112-4515-9d83-cd6ce04d96b6 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:23.159332 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af6867-867f-4f79-9009-7574fef09248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:23.160015 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af6867-867f-4f79-9009-7574fef09248 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:23.160299 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af6867-867f-4f79-9009-7574fef09248 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:23.160574 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af6867-867f-4f79-9009-7574fef09248 None admin] PATH_INFO: `/v3/projects/95a15538272e426ba99c0b50e2f30f52` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:23.161726 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14af6867-867f-4f79-9009-7574fef09248 None admin] RBAC: Authorizing `identity:get_project(project_id=95a15538272e426ba99c0b50e2f30f52)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:23.163187 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14af6867-867f-4f79-9009-7574fef09248 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:23.165939 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 148/287] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:22 2022] GET /identity/v3/projects/95a15538272e426ba99c0b50e2f30f52 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:23.171663 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f60dc9-481d-46c9-956b-bf7165a4c42c None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:23.317681 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:23.320191 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af6867-867f-4f79-9009-7574fef09248 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:23.326076 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f60dc9-481d-46c9-956b-bf7165a4c42c None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:23.361811 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08aa886-5a44-48ad-95a0-21c0ccc96c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:23.362540 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08aa886-5a44-48ad-95a0-21c0ccc96c7b None admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:23.362926 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08aa886-5a44-48ad-95a0-21c0ccc96c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:23.363209 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08aa886-5a44-48ad-95a0-21c0ccc96c7b None admin] PATH_INFO: `/v3/projects/95a15538272e426ba99c0b50e2f30f52/users/98f73a46b3254849be8c8d175b2ac40c/roles/4be260163ba342e9b42232a49eec9fec` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:23.365278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08aa886-5a44-48ad-95a0-21c0ccc96c7b None admin] RBAC: Authorizing `identity:create_grant(project_id=95a15538272e426ba99c0b50e2f30f52, user_id=98f73a46b3254849be8c8d175b2ac40c, role_id=4be260163ba342e9b42232a49eec9fec)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:23.367026 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08aa886-5a44-48ad-95a0-21c0ccc96c7b None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:23.388114 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 140/288] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Aug 23 04:14:23 2022] PUT /identity/v3/projects/95a15538272e426ba99c0b50e2f30f52/users/98f73a46b3254849be8c8d175b2ac40c/roles/4be260163ba342e9b42232a49eec9fec => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:14:23.496612 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af6867-867f-4f79-9009-7574fef09248 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:23.548840 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:23.557480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04471945-84e2-40d1-a14c-be5ae7396ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:23.558321 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04471945-84e2-40d1-a14c-be5ae7396ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:23.558738 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04471945-84e2-40d1-a14c-be5ae7396ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:23.559178 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04471945-84e2-40d1-a14c-be5ae7396ae5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:23.575660 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04471945-84e2-40d1-a14c-be5ae7396ae5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:23.581923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-04471945-84e2-40d1-a14c-be5ae7396ae5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:23.581923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:23.581923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:23.581923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:23.581923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:23.581923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:23.581923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:23.581923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:23.581923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:23.581923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:23.581923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:23.581923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:23.581923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:23.581923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:23.581923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:23.581923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:23.583055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:23.583055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:23.583055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:23.583055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:23.583055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:23.583055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:23.583055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:23.583055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:23.583055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:23.583055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:23.583055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:23.583055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:23.583055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:23.583055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:23.583055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:23.583055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:23.584031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:23.584031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:23.584031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:23.584031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:23.584031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:23.584031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:23.584031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:23.584031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:23.584031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:23.584031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:23.584031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:23.584031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:23.586976 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 149/289] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:14:23.593943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08aa886-5a44-48ad-95a0-21c0ccc96c7b None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:23.706477 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08aa886-5a44-48ad-95a0-21c0ccc96c7b None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:23.747475 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b04219-b2fa-4d3b-a677-a490b3987665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:23.748351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b04219-b2fa-4d3b-a677-a490b3987665 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:23.748771 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b04219-b2fa-4d3b-a677-a490b3987665 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:23.749209 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b04219-b2fa-4d3b-a677-a490b3987665 None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:23.750018 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b04219-b2fa-4d3b-a677-a490b3987665 None admin] RBAC: Adding query filter params (name=member) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:23.750460 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b04219-b2fa-4d3b-a677-a490b3987665 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:23.751998 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b04219-b2fa-4d3b-a677-a490b3987665 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:23.763584 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 141/290] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Aug 23 04:14:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:14:23.769517 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04471945-84e2-40d1-a14c-be5ae7396ae5 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:23.911714 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04471945-84e2-40d1-a14c-be5ae7396ae5 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:23.966668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e52aa8-9153-4d01-8302-34b4fa1ca33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:23.967326 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e52aa8-9153-4d01-8302-34b4fa1ca33c None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:23.967609 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e52aa8-9153-4d01-8302-34b4fa1ca33c None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:23.967885 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e52aa8-9153-4d01-8302-34b4fa1ca33c None admin] PATH_INFO: `/v3/users/4772005528a1418a904ae1d87d9e0f18` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:23.969098 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e52aa8-9153-4d01-8302-34b4fa1ca33c None admin] RBAC: Authorizing `identity:get_user(user_id=4772005528a1418a904ae1d87d9e0f18)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:23.970569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e52aa8-9153-4d01-8302-34b4fa1ca33c None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:23.972523 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 150/291] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:14:23 2022] GET /identity/v3/users/4772005528a1418a904ae1d87d9e0f18 => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:23.978693 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b04219-b2fa-4d3b-a677-a490b3987665 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:24.024290 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:24.142215 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b04219-b2fa-4d3b-a677-a490b3987665 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:24.187455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6068ba8-c94d-4234-9620-73bd5442c4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:24.188100 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6068ba8-c94d-4234-9620-73bd5442c4ce None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:24.188323 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6068ba8-c94d-4234-9620-73bd5442c4ce None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:24.188544 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6068ba8-c94d-4234-9620-73bd5442c4ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:24.258440 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6068ba8-c94d-4234-9620-73bd5442c4ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:24.259899 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6068ba8-c94d-4234-9620-73bd5442c4ce None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:24.284277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:24.381513 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 142/292] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:14:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:14:24.389653 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e52aa8-9153-4d01-8302-34b4fa1ca33c None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:24.486096 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:24.557833 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c73e11-5879-406f-a300-79430ea3da4a None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:24.558179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c73e11-5879-406f-a300-79430ea3da4a None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:24.558412 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c73e11-5879-406f-a300-79430ea3da4a None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:24.563471 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 143/293] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:14:24.564689 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ed304-985c-4794-a5e0-c2dc400e3157 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:24.564976 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ed304-985c-4794-a5e0-c2dc400e3157 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:24.565244 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ed304-985c-4794-a5e0-c2dc400e3157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:24.568589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e52aa8-9153-4d01-8302-34b4fa1ca33c None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:24.584011 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-0d0ed304-985c-4794-a5e0-c2dc400e3157 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:24.585207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-0d0ed304-985c-4794-a5e0-c2dc400e3157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:24.614442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0981c0de-f271-4f28-8159-e9d53f3b57c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:24.615077 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0981c0de-f271-4f28-8159-e9d53f3b57c9 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:24.615349 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0981c0de-f271-4f28-8159-e9d53f3b57c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:24.615696 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0981c0de-f271-4f28-8159-e9d53f3b57c9 None admin] PATH_INFO: `/v3/projects/95a15538272e426ba99c0b50e2f30f52` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:24.616966 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0981c0de-f271-4f28-8159-e9d53f3b57c9 None admin] RBAC: Authorizing `identity:get_project(project_id=95a15538272e426ba99c0b50e2f30f52)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:24.618458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0981c0de-f271-4f28-8159-e9d53f3b57c9 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:24.620749 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 151/294] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:24 2022] GET /identity/v3/projects/95a15538272e426ba99c0b50e2f30f52 => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:24.621978 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6c2a8-940f-4677-9580-b4726b61fabb None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:24.622259 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6c2a8-940f-4677-9580-b4726b61fabb None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:24.622562 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6c2a8-940f-4677-9580-b4726b61fabb None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:24.623801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 152/295] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:24.626871 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad6c2a8-940f-4677-9580-b4726b61fabb None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:24.676234 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 144/296] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 04:14:24.677511 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07703a6-b222-4f87-a2d8-a96d3523a1cd None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:24.680170 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07703a6-b222-4f87-a2d8-a96d3523a1cd None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:24.680465 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07703a6-b222-4f87-a2d8-a96d3523a1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:24.711838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-f07703a6-b222-4f87-a2d8-a96d3523a1cd None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:24.712975 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-f07703a6-b222-4f87-a2d8-a96d3523a1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:24.760305 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad6c2a8-940f-4677-9580-b4726b61fabb None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:24.776753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:24.797120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:24.825383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 145/297] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:24.833673 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30479a8b-7bb4-443d-a280-ec17fa3c77e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:24.834367 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30479a8b-7bb4-443d-a280-ec17fa3c77e6 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:24.834644 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30479a8b-7bb4-443d-a280-ec17fa3c77e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:24.834912 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30479a8b-7bb4-443d-a280-ec17fa3c77e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:24.835718 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30479a8b-7bb4-443d-a280-ec17fa3c77e6 None admin] RBAC: Adding query filter params (role.id=aa03504b4fb642d288d07f4335e291e1, scope.project.id=95a15538272e426ba99c0b50e2f30f52, user.id=4772005528a1418a904ae1d87d9e0f18) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:24.836048 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30479a8b-7bb4-443d-a280-ec17fa3c77e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:24.837502 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30479a8b-7bb4-443d-a280-ec17fa3c77e6 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:24.849180 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 153/298] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:14:24 2022] GET /identity/v3/role_assignments?user.id=4772005528a1418a904ae1d87d9e0f18&scope.project.id=95a15538272e426ba99c0b50e2f30f52&role.id=aa03504b4fb642d288d07f4335e291e1 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:24.930543 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc33304-a8b8-4e46-825d-b4f5bfb910b0 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:24.930543 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc33304-a8b8-4e46-825d-b4f5bfb910b0 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:24.930543 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc33304-a8b8-4e46-825d-b4f5bfb910b0 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:24.930543 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 146/299] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:24.939974 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7555bb9-86ca-4e77-8aa6-92f43a55a315 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:24.939974 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7555bb9-86ca-4e77-8aa6-92f43a55a315 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:24.939974 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7555bb9-86ca-4e77-8aa6-92f43a55a315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:24.964865 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-c7555bb9-86ca-4e77-8aa6-92f43a55a315 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:24.966400 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-c7555bb9-86ca-4e77-8aa6-92f43a55a315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:25.035340 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc33304-a8b8-4e46-825d-b4f5bfb910b0 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:25.069032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 154/300] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:25.149121 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc33304-a8b8-4e46-825d-b4f5bfb910b0 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:25.184904 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3014b59b-fc13-4a22-926b-fa5913025ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:25.185774 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014b59b-fc13-4a22-926b-fa5913025ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:25.186192 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014b59b-fc13-4a22-926b-fa5913025ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:25.186633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014b59b-fc13-4a22-926b-fa5913025ec1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:25.193331 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7555bb9-86ca-4e77-8aa6-92f43a55a315 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:25.199796 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3014b59b-fc13-4a22-926b-fa5913025ec1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:25.205319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-3014b59b-fc13-4a22-926b-fa5913025ec1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:25.205319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:25.205319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:25.205319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:25.205319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:25.205319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:25.205319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:25.205319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:25.205319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:25.205319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:25.205319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:25.205319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:25.205319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:25.205319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:25.205319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:25.205319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:25.206192 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:25.206192 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:25.206192 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:25.206192 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:25.206192 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:25.206192 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:25.206192 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:25.206192 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:25.206192 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:25.206192 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:25.206192 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:25.206192 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:25.206192 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:25.206192 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:25.206192 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:25.206192 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:25.206947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:25.206947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:25.206947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:25.206947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:25.206947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:25.206947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:25.206947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:25.206947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:25.206947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:25.206947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:25.206947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:25.206947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:25.209480 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 147/301] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:25.215989 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3014b59b-fc13-4a22-926b-fa5913025ec1 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:25.255670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:25.321989 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7555bb9-86ca-4e77-8aa6-92f43a55a315 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:25.330267 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3014b59b-fc13-4a22-926b-fa5913025ec1 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:25.331261 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:25.368536 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2cfa78-116c-454a-9d2a-58bcb7ff2ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:25.369334 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2cfa78-116c-454a-9d2a-58bcb7ff2ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:25.369614 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2cfa78-116c-454a-9d2a-58bcb7ff2ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:25.369883 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2cfa78-116c-454a-9d2a-58bcb7ff2ca0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:25.372821 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7f710-25fe-419e-98f1-658bc95a3455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:25.373451 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7f710-25fe-419e-98f1-658bc95a3455 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:25.373727 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7f710-25fe-419e-98f1-658bc95a3455 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:25.373994 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7f710-25fe-419e-98f1-658bc95a3455 None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:25.374665 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7f710-25fe-419e-98f1-658bc95a3455 None admin] RBAC: Adding query filter params (name=member) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:25.374958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7f710-25fe-419e-98f1-658bc95a3455 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:25.376454 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7f710-25fe-419e-98f1-658bc95a3455 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:25.385983 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 148/302] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Aug 23 04:14:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:25.386644 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7f710-25fe-419e-98f1-658bc95a3455 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:25.391725 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2cfa78-116c-454a-9d2a-58bcb7ff2ca0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:25.396543 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-1e2cfa78-116c-454a-9d2a-58bcb7ff2ca0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 04:14:25.396543 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:25.396543 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:25.396543 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:25.396543 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:25.396543 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:25.396543 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:25.396543 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:25.396543 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:25.396543 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:25.396543 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:25.396543 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:25.396543 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:25.396543 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:25.396543 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:25.396543 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:25.397382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:25.397382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:25.397382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:25.397382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:25.397382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:25.397382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:25.397382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:25.397382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:25.397382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:25.397382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:25.397382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:25.397382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:25.397382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:25.397382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:25.397382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:25.397382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:25.398198 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:25.398198 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:25.398198 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:25.398198 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:25.398198 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:25.398198 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:25.398198 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:25.398198 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:25.398198 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:25.398198 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:25.398198 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 04:14:25.398198 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:25.398198 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 155/303] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:14:25.398960 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2cfa78-116c-454a-9d2a-58bcb7ff2ca0 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:25.503562 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:25.507550 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7f710-25fe-419e-98f1-658bc95a3455 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:25.508185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2cfa78-116c-454a-9d2a-58bcb7ff2ca0 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:25.543935 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082b20d-d879-462a-bd7d-85fc16c8aa5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:25.544615 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082b20d-d879-462a-bd7d-85fc16c8aa5a None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:25.544863 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082b20d-d879-462a-bd7d-85fc16c8aa5a None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:25.545179 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082b20d-d879-462a-bd7d-85fc16c8aa5a None admin] PATH_INFO: `/v3/users/78e7881aab9b46fda160b9662233c76e` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:25.546243 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a082b20d-d879-462a-bd7d-85fc16c8aa5a None admin] RBAC: Authorizing `identity:get_user(user_id=78e7881aab9b46fda160b9662233c76e)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:25.546809 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf00155-12a7-4f59-a817-285bffe3eaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:25.547468 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf00155-12a7-4f59-a817-285bffe3eaa0 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:25.547716 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a082b20d-d879-462a-bd7d-85fc16c8aa5a None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:25.547942 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf00155-12a7-4f59-a817-285bffe3eaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:25.548428 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf00155-12a7-4f59-a817-285bffe3eaa0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:25.549653 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 156/304] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:14:25 2022] GET /identity/v3/users/78e7881aab9b46fda160b9662233c76e => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:25.549859 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf00155-12a7-4f59-a817-285bffe3eaa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:25.551289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf00155-12a7-4f59-a817-285bffe3eaa0 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:25.553004 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 149/305] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:14:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:25.555010 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082b20d-d879-462a-bd7d-85fc16c8aa5a None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:25.558158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf00155-12a7-4f59-a817-285bffe3eaa0 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:25.600768 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:25.669826 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf00155-12a7-4f59-a817-285bffe3eaa0 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:25.676987 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082b20d-d879-462a-bd7d-85fc16c8aa5a None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:25.706022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e47312-6071-4692-a7e4-03d2096fa7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:25.706657 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e47312-6071-4692-a7e4-03d2096fa7cf None admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:25.706928 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e47312-6071-4692-a7e4-03d2096fa7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:25.707313 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e47312-6071-4692-a7e4-03d2096fa7cf None admin] PATH_INFO: `/v3/system/users/78e7881aab9b46fda160b9662233c76e/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:25.708596 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e47312-6071-4692-a7e4-03d2096fa7cf None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=78e7881aab9b46fda160b9662233c76e, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:25.709978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e47312-6071-4692-a7e4-03d2096fa7cf None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:25.712497 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30361664-7b82-4f99-a236-f4a2f683b5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:25.713124 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30361664-7b82-4f99-a236-f4a2f683b5e1 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:25.713404 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30361664-7b82-4f99-a236-f4a2f683b5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:25.714030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30361664-7b82-4f99-a236-f4a2f683b5e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:25.714794 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30361664-7b82-4f99-a236-f4a2f683b5e1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:25.715081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30361664-7b82-4f99-a236-f4a2f683b5e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:25.716771 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30361664-7b82-4f99-a236-f4a2f683b5e1 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:25.720386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 150/306] 10.222.0.18 () {58 vars in 1334 bytes} [Tue Aug 23 04:14:25 2022] PUT /identity/v3/system/users/78e7881aab9b46fda160b9662233c76e/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:14:25.726284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 157/307] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Aug 23 04:14:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:25.727596 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae078b4-36cd-442c-801d-5bfcca8f274f None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:25.727886 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae078b4-36cd-442c-801d-5bfcca8f274f None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:25.728197 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae078b4-36cd-442c-801d-5bfcca8f274f None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:25.729704 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 158/308] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:25.732881 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e47312-6071-4692-a7e4-03d2096fa7cf None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:25.733207 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae078b4-36cd-442c-801d-5bfcca8f274f None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:25.773652 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:25.841832 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:25.849176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae078b4-36cd-442c-801d-5bfcca8f274f None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:25.861299 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e47312-6071-4692-a7e4-03d2096fa7cf None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:25.889351 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c824d41-4bb4-4e8d-b196-02b15459b155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:25.889990 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c824d41-4bb4-4e8d-b196-02b15459b155 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:25.890375 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c824d41-4bb4-4e8d-b196-02b15459b155 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:25.890647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c824d41-4bb4-4e8d-b196-02b15459b155 None admin] PATH_INFO: `/v3/users/03e2250f56cc4006a687c702c65b78ea` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:25.899091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0134fb-f9ec-4eb6-8b33-13dd0844a51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:25.899711 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0134fb-f9ec-4eb6-8b33-13dd0844a51a None admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:25.900019 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0134fb-f9ec-4eb6-8b33-13dd0844a51a None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:25.900288 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0134fb-f9ec-4eb6-8b33-13dd0844a51a None admin] PATH_INFO: `/v3/groups` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:25.900913 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0134fb-f9ec-4eb6-8b33-13dd0844a51a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:25.902358 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0134fb-f9ec-4eb6-8b33-13dd0844a51a None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:25.906491 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c824d41-4bb4-4e8d-b196-02b15459b155 None admin] RBAC: Authorizing `identity:get_user(user_id=03e2250f56cc4006a687c702c65b78ea)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:25.908336 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c824d41-4bb4-4e8d-b196-02b15459b155 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:25.910201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 159/309] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:14:25 2022] GET /identity/v3/users/03e2250f56cc4006a687c702c65b78ea => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:25.911630 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb64106-2e9d-4961-b07c-62acc371de2b None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:25.911935 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb64106-2e9d-4961-b07c-62acc371de2b None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:25.912198 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb64106-2e9d-4961-b07c-62acc371de2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:25.923754 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 151/310] 10.222.0.18 () {60 vars in 1134 bytes} [Tue Aug 23 04:14:25 2022] POST /identity/v3/groups => generated 229 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:14:25.924843 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0134fb-f9ec-4eb6-8b33-13dd0844a51a None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:25.935284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-3fb64106-2e9d-4961-b07c-62acc371de2b None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:25.936561 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-3fb64106-2e9d-4961-b07c-62acc371de2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:26.033356 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 160/311] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:26.040513 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:26.051565 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0134fb-f9ec-4eb6-8b33-13dd0844a51a None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:26.091344 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9196096-915e-4506-a58c-391ea0cc2db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:26.092237 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9196096-915e-4506-a58c-391ea0cc2db4 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:26.092661 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9196096-915e-4506-a58c-391ea0cc2db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:26.093299 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9196096-915e-4506-a58c-391ea0cc2db4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:26.094063 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9196096-915e-4506-a58c-391ea0cc2db4 None admin] RBAC: Adding query filter params (role.id=4be260163ba342e9b42232a49eec9fec, user.id=03e2250f56cc4006a687c702c65b78ea, scope.system=all) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:26.094372 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9196096-915e-4506-a58c-391ea0cc2db4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:26.095828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9196096-915e-4506-a58c-391ea0cc2db4 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:26.101051 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 152/312] 10.222.0.18 () {56 vars in 1308 bytes} [Tue Aug 23 04:14:25 2022] GET /identity/v3/role_assignments?user.id=03e2250f56cc4006a687c702c65b78ea&scope.system=all&role.id=4be260163ba342e9b42232a49eec9fec => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:26.306088 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb64106-2e9d-4961-b07c-62acc371de2b None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:26.393431 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:26.411145 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb64106-2e9d-4961-b07c-62acc371de2b None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:26.452162 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb115c2-6597-4211-8e11-f8ab2035f5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:26.453171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb115c2-6597-4211-8e11-f8ab2035f5bb None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:26.453675 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb115c2-6597-4211-8e11-f8ab2035f5bb None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:26.454157 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb115c2-6597-4211-8e11-f8ab2035f5bb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:26.466605 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb115c2-6597-4211-8e11-f8ab2035f5bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:26.471171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-ebb115c2-6597-4211-8e11-f8ab2035f5bb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 04:14:26.471171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:26.471171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:26.471171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:26.471171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:26.471171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:26.471171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:26.471171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:26.471171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:26.471171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:26.471171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:26.471171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:26.471171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:26.471171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:26.471171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:26.471171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:26.472045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:26.472045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:26.472045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:26.472045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:26.472045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:26.472045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:26.472045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:26.472045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:26.472045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:26.472045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:26.472045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:26.472045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:26.472045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:26.472045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:26.472045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:26.472045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:26.472827 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:26.472827 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:26.472827 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:26.472827 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:26.472827 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:26.472827 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:26.472827 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:26.472827 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:26.472827 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:26.472827 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:26.472827 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 04:14:26.472827 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:26.472827 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 161/313] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:26.478284 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9196096-915e-4506-a58c-391ea0cc2db4 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:26.605526 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9196096-915e-4506-a58c-391ea0cc2db4 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:26.641465 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d52c9-a2c8-4f44-95ef-77b03bede49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:26.642135 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d52c9-a2c8-4f44-95ef-77b03bede49d None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:26.642416 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d52c9-a2c8-4f44-95ef-77b03bede49d None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:26.642684 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d52c9-a2c8-4f44-95ef-77b03bede49d None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:26.643354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606d52c9-a2c8-4f44-95ef-77b03bede49d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:26.643656 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606d52c9-a2c8-4f44-95ef-77b03bede49d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:26.645054 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606d52c9-a2c8-4f44-95ef-77b03bede49d None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:26.654207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 153/314] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Aug 23 04:14:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:26.659722 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb115c2-6597-4211-8e11-f8ab2035f5bb None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:26.762057 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb115c2-6597-4211-8e11-f8ab2035f5bb None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:26.799310 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac57f187-d6ea-4c90-8e36-977295fcab86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:26.799972 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57f187-d6ea-4c90-8e36-977295fcab86 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:26.800259 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57f187-d6ea-4c90-8e36-977295fcab86 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:26.800534 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57f187-d6ea-4c90-8e36-977295fcab86 None admin] PATH_INFO: `/v3/users/98f73a46b3254849be8c8d175b2ac40c` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:26.801671 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac57f187-d6ea-4c90-8e36-977295fcab86 None admin] RBAC: Authorizing `identity:get_user(user_id=98f73a46b3254849be8c8d175b2ac40c)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:26.803130 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac57f187-d6ea-4c90-8e36-977295fcab86 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:26.805102 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 162/315] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:14:26 2022] GET /identity/v3/users/98f73a46b3254849be8c8d175b2ac40c => generated 297 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:26.810484 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d52c9-a2c8-4f44-95ef-77b03bede49d None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:26.830543 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:26.918894 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d52c9-a2c8-4f44-95ef-77b03bede49d None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:26.957153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28466206-b48d-44eb-adc8-79b3785fcfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:26.957761 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28466206-b48d-44eb-adc8-79b3785fcfcd None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:26.958036 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28466206-b48d-44eb-adc8-79b3785fcfcd None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:26.958304 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28466206-b48d-44eb-adc8-79b3785fcfcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:27.027627 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28466206-b48d-44eb-adc8-79b3785fcfcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:27.029126 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28466206-b48d-44eb-adc8-79b3785fcfcd None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:27.046678 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:27.131790 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 154/316] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:14:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:14:27.137700 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac57f187-d6ea-4c90-8e36-977295fcab86 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:27.153297 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:27.245107 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac57f187-d6ea-4c90-8e36-977295fcab86 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:27.281473 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76ce7e-d1fd-45fc-9990-346908fd1965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:27.282120 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76ce7e-d1fd-45fc-9990-346908fd1965 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:27.282462 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76ce7e-d1fd-45fc-9990-346908fd1965 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:27.282737 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76ce7e-d1fd-45fc-9990-346908fd1965 None admin] PATH_INFO: `/v3/projects/95a15538272e426ba99c0b50e2f30f52` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:27.283855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76ce7e-d1fd-45fc-9990-346908fd1965 None admin] RBAC: Authorizing `identity:get_project(project_id=95a15538272e426ba99c0b50e2f30f52)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:27.285325 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76ce7e-d1fd-45fc-9990-346908fd1965 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:27.286955 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 163/317] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:27 2022] GET /identity/v3/projects/95a15538272e426ba99c0b50e2f30f52 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:27.293322 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28466206-b48d-44eb-adc8-79b3785fcfcd None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:27.393962 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28466206-b48d-44eb-adc8-79b3785fcfcd None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:27.411263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:27.429391 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d089992-8db3-4173-b844-508e33e8f6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:27.430081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d089992-8db3-4173-b844-508e33e8f6f3 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:27.430356 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d089992-8db3-4173-b844-508e33e8f6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:27.430622 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d089992-8db3-4173-b844-508e33e8f6f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:27.431348 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d089992-8db3-4173-b844-508e33e8f6f3 None admin] RBAC: Adding query filter params (role.id=4be260163ba342e9b42232a49eec9fec, scope.project.id=95a15538272e426ba99c0b50e2f30f52, user.id=98f73a46b3254849be8c8d175b2ac40c) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:27.431650 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d089992-8db3-4173-b844-508e33e8f6f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:27.433132 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d089992-8db3-4173-b844-508e33e8f6f3 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:27.439364 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 155/318] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:14:27 2022] GET /identity/v3/role_assignments?user.id=98f73a46b3254849be8c8d175b2ac40c&scope.project.id=95a15538272e426ba99c0b50e2f30f52&role.id=4be260163ba342e9b42232a49eec9fec => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:27.722090 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a88a6e-5ccc-444b-917b-599a4c4deca9 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:27.722616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a88a6e-5ccc-444b-917b-599a4c4deca9 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:27.722616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a88a6e-5ccc-444b-917b-599a4c4deca9 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:27.724167 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 164/319] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:27.729597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f26339-54b1-4a66-8111-9f382e496e88 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:27.730031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f26339-54b1-4a66-8111-9f382e496e88 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:27.730100 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f26339-54b1-4a66-8111-9f382e496e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:27.748811 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-c8f26339-54b1-4a66-8111-9f382e496e88 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:27.750004 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-c8f26339-54b1-4a66-8111-9f382e496e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:27.819976 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 156/320] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 04:14:27.962512 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c40586-d6eb-461c-9f5d-c64cd6454ceb None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:27.962749 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c40586-d6eb-461c-9f5d-c64cd6454ceb None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:27.963016 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c40586-d6eb-461c-9f5d-c64cd6454ceb None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:27.964959 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 165/321] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:27.970132 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e71e0-4e00-46e8-82b4-136d33b7c606 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:27.970422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e71e0-4e00-46e8-82b4-136d33b7c606 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:27.970690 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e71e0-4e00-46e8-82b4-136d33b7c606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:27.989781 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-4d1e71e0-4e00-46e8-82b4-136d33b7c606 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:27.990895 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-4d1e71e0-4e00-46e8-82b4-136d33b7c606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:28.059960 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 157/322] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:28.085766 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c40586-d6eb-461c-9f5d-c64cd6454ceb None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:28.103266 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac5704-059f-4ca1-a04b-87b9859d6594 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:28.103544 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac5704-059f-4ca1-a04b-87b9859d6594 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:28.103820 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac5704-059f-4ca1-a04b-87b9859d6594 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:28.105334 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 158/323] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:14:28.110114 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06765df-040e-46f2-9bf7-2b0f243dac38 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:28.110396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06765df-040e-46f2-9bf7-2b0f243dac38 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:28.110664 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06765df-040e-46f2-9bf7-2b0f243dac38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:28.128852 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-b06765df-040e-46f2-9bf7-2b0f243dac38 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:28.130024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-b06765df-040e-46f2-9bf7-2b0f243dac38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:28.168879 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:28.184348 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:28.203883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 159/324] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 04:14:28.206402 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c40586-d6eb-461c-9f5d-c64cd6454ceb None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:28.243343 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a136c1-56ea-41b8-bc9c-ba3ac5286d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:28.243906 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a136c1-56ea-41b8-bc9c-ba3ac5286d53 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:28.244211 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a136c1-56ea-41b8-bc9c-ba3ac5286d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:28.244480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a136c1-56ea-41b8-bc9c-ba3ac5286d53 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:28.254724 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a136c1-56ea-41b8-bc9c-ba3ac5286d53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:28.260023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-40a136c1-56ea-41b8-bc9c-ba3ac5286d53 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:28.260023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:28.260023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:28.260023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:28.260023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:28.260023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:28.260023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:28.260023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:28.260023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:28.260023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:28.260023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:28.260023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:28.260023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:28.260023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:28.260023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:28.260023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:28.260884 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:28.260884 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:28.260884 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:28.260884 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:28.260884 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:28.260884 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:28.260884 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:28.260884 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:28.260884 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:28.260884 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:28.260884 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:28.260884 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:28.260884 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:28.260884 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:28.260884 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:28.260884 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:28.261837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:28.261837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:28.261837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:28.261837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:28.261837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:28.261837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:28.261837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:28.261837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:28.261837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:28.261837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:28.261837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:28.261837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:28.261837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 166/325] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:28.269227 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06765df-040e-46f2-9bf7-2b0f243dac38 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:28.321057 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a136c1-56ea-41b8-bc9c-ba3ac5286d53 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:28.371973 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06765df-040e-46f2-9bf7-2b0f243dac38 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:28.409794 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742e4fc-32ca-4f7a-9bc1-547164baae1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:28.410716 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742e4fc-32ca-4f7a-9bc1-547164baae1c None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:28.411165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742e4fc-32ca-4f7a-9bc1-547164baae1c None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:28.411701 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742e4fc-32ca-4f7a-9bc1-547164baae1c None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:28.412701 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742e4fc-32ca-4f7a-9bc1-547164baae1c None admin] RBAC: Adding query filter params (name=member) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:28.413207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742e4fc-32ca-4f7a-9bc1-547164baae1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:28.415482 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742e4fc-32ca-4f7a-9bc1-547164baae1c None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:28.425359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 160/326] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Aug 23 04:14:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:28.430737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742e4fc-32ca-4f7a-9bc1-547164baae1c None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:28.432898 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a136c1-56ea-41b8-bc9c-ba3ac5286d53 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:28.468047 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:28.477384 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66963618-e472-4560-82cf-d9cd68bd188e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:28.478035 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66963618-e472-4560-82cf-d9cd68bd188e None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:28.478316 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66963618-e472-4560-82cf-d9cd68bd188e None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:28.478588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66963618-e472-4560-82cf-d9cd68bd188e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:28.479718 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66963618-e472-4560-82cf-d9cd68bd188e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:28.481213 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66963618-e472-4560-82cf-d9cd68bd188e None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:28.483985 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 167/327] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:14:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:14:28.483985 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66963618-e472-4560-82cf-d9cd68bd188e None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:28.498164 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:28.542057 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742e4fc-32ca-4f7a-9bc1-547164baae1c None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:28.576893 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c20c8-cf59-4d30-a4aa-5a2c84f42b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:28.577530 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c20c8-cf59-4d30-a4aa-5a2c84f42b5e None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:28.577811 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c20c8-cf59-4d30-a4aa-5a2c84f42b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:28.578078 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c20c8-cf59-4d30-a4aa-5a2c84f42b5e None admin] PATH_INFO: `/v3/users/78e7881aab9b46fda160b9662233c76e` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:28.579257 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c20c8-cf59-4d30-a4aa-5a2c84f42b5e None admin] RBAC: Authorizing `identity:get_user(user_id=78e7881aab9b46fda160b9662233c76e)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:28.580723 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c20c8-cf59-4d30-a4aa-5a2c84f42b5e None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:28.582307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 161/328] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:14:28 2022] GET /identity/v3/users/78e7881aab9b46fda160b9662233c76e => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:28.583078 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c20c8-cf59-4d30-a4aa-5a2c84f42b5e None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:28.589904 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66963618-e472-4560-82cf-d9cd68bd188e None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:28.625932 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9be01-e67d-4381-bfda-1a9b609723a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:28.626565 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9be01-e67d-4381-bfda-1a9b609723a9 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:28.626842 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9be01-e67d-4381-bfda-1a9b609723a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:28.627115 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9be01-e67d-4381-bfda-1a9b609723a9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:28.637751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9be01-e67d-4381-bfda-1a9b609723a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:28.642837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-c9f9be01-e67d-4381-bfda-1a9b609723a9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 04:14:28.642837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:28.642837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:28.642837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:28.642837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:28.642837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:28.642837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:28.642837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:28.642837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:28.642837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:28.642837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:28.642837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:28.642837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:28.642837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:28.642837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:28.642837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:28.643689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:28.643689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:28.643689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:28.643689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:28.643689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:28.643689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:28.643689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:28.643689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:28.643689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:28.643689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:28.643689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:28.643689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:28.643689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:28.643689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:28.643689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:28.643689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:28.644509 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:28.644509 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:28.644509 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:28.644509 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:28.644509 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:28.644509 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:28.644509 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:28.644509 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:28.644509 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:28.644509 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:28.644509 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 04:14:28.644509 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:28.644509 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 168/329] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:28.645131 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9be01-e67d-4381-bfda-1a9b609723a9 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:28.686238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:28.689466 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c20c8-cf59-4d30-a4aa-5a2c84f42b5e None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:28.717186 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:28.724285 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6254e05-556f-423c-9ca9-1813d408dfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:28.724899 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6254e05-556f-423c-9ca9-1813d408dfc6 None admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:28.725171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6254e05-556f-423c-9ca9-1813d408dfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:28.725439 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6254e05-556f-423c-9ca9-1813d408dfc6 None admin] PATH_INFO: `/v3/groups` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:28.726065 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6254e05-556f-423c-9ca9-1813d408dfc6 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:28.727495 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6254e05-556f-423c-9ca9-1813d408dfc6 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:28.743220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 162/330] 10.222.0.18 () {60 vars in 1134 bytes} [Tue Aug 23 04:14:28 2022] POST /identity/v3/groups => generated 226 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:14:28.749999 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6254e05-556f-423c-9ca9-1813d408dfc6 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:28.753432 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9be01-e67d-4381-bfda-1a9b609723a9 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:28.792671 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a3100-1187-47c8-9538-f6b43292ca6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:28.793303 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a3100-1187-47c8-9538-f6b43292ca6b None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:28.793577 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a3100-1187-47c8-9538-f6b43292ca6b None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:28.793842 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a3100-1187-47c8-9538-f6b43292ca6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:28.794590 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a3100-1187-47c8-9538-f6b43292ca6b None admin] RBAC: Adding query filter params (role.id=aa03504b4fb642d288d07f4335e291e1, user.id=78e7881aab9b46fda160b9662233c76e, scope.system=all) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:28.794896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a3100-1187-47c8-9538-f6b43292ca6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:28.796470 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a3100-1187-47c8-9538-f6b43292ca6b None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:28.810224 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 169/331] 10.222.0.18 () {56 vars in 1308 bytes} [Tue Aug 23 04:14:28 2022] GET /identity/v3/role_assignments?user.id=78e7881aab9b46fda160b9662233c76e&scope.system=all&role.id=aa03504b4fb642d288d07f4335e291e1 => generated 510 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:28.875555 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6254e05-556f-423c-9ca9-1813d408dfc6 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:28.921335 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed6ece-af6d-487b-8853-4af7bbc7f454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:28.921556 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed6ece-af6d-487b-8853-4af7bbc7f454 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:28.921799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed6ece-af6d-487b-8853-4af7bbc7f454 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:28.922065 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed6ece-af6d-487b-8853-4af7bbc7f454 None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:28.925089 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed6ece-af6d-487b-8853-4af7bbc7f454 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:28.925089 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed6ece-af6d-487b-8853-4af7bbc7f454 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:28.925089 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed6ece-af6d-487b-8853-4af7bbc7f454 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:28.936062 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 163/332] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Aug 23 04:14:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:28.941835 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a3100-1187-47c8-9538-f6b43292ca6b None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:29.052150 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a3100-1187-47c8-9538-f6b43292ca6b None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:29.081183 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:29.090953 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3dfe4-7f99-42db-8de5-ab0e0e9062ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:29.091593 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3dfe4-7f99-42db-8de5-ab0e0e9062ac None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:29.091870 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3dfe4-7f99-42db-8de5-ab0e0e9062ac None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:29.092222 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3dfe4-7f99-42db-8de5-ab0e0e9062ac None admin] PATH_INFO: `/v3/users/03e2250f56cc4006a687c702c65b78ea` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:29.093354 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e3dfe4-7f99-42db-8de5-ab0e0e9062ac None admin] RBAC: Authorizing `identity:get_user(user_id=03e2250f56cc4006a687c702c65b78ea)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:29.094812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e3dfe4-7f99-42db-8de5-ab0e0e9062ac None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:29.096437 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 170/333] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:14:28 2022] GET /identity/v3/users/03e2250f56cc4006a687c702c65b78ea => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:29.102641 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed6ece-af6d-487b-8853-4af7bbc7f454 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:29.129661 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:29.218555 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed6ece-af6d-487b-8853-4af7bbc7f454 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:29.256621 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293049a9-a013-4e78-a415-25e8917828fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:29.257585 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293049a9-a013-4e78-a415-25e8917828fd None admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:29.258005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293049a9-a013-4e78-a415-25e8917828fd None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:29.258420 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293049a9-a013-4e78-a415-25e8917828fd None admin] PATH_INFO: `/v3/system/users/03e2250f56cc4006a687c702c65b78ea/roles/4be260163ba342e9b42232a49eec9fec` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:29.260038 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293049a9-a013-4e78-a415-25e8917828fd None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=03e2250f56cc4006a687c702c65b78ea, role_id=4be260163ba342e9b42232a49eec9fec)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:29.261666 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293049a9-a013-4e78-a415-25e8917828fd None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:29.269792 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 164/334] 10.222.0.18 () {58 vars in 1334 bytes} [Tue Aug 23 04:14:29 2022] PUT /identity/v3/system/users/03e2250f56cc4006a687c702c65b78ea/roles/4be260163ba342e9b42232a49eec9fec => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:14:31.122327 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b53a7-b49e-4089-9505-d098796a01ec None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:31.122949 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b53a7-b49e-4089-9505-d098796a01ec None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:31.122949 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b53a7-b49e-4089-9505-d098796a01ec None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:31.124537 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 171/335] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:31.130638 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088f67e-1cde-4157-b00d-23679d7aa11b None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:31.131046 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088f67e-1cde-4157-b00d-23679d7aa11b None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:31.131253 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088f67e-1cde-4157-b00d-23679d7aa11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:31.156929 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-4088f67e-1cde-4157-b00d-23679d7aa11b None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:31.158137 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-4088f67e-1cde-4157-b00d-23679d7aa11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:31.240913 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 165/336] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 04:14:31.291084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55381ad4-e2ee-4e32-8517-8e519c9e25e8 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:31.291422 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55381ad4-e2ee-4e32-8517-8e519c9e25e8 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:31.291633 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55381ad4-e2ee-4e32-8517-8e519c9e25e8 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:31.293051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 172/337] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:31.308773 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03937287-9e9c-44e5-8df1-cb6ae06b6856 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:31.309277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03937287-9e9c-44e5-8df1-cb6ae06b6856 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:31.309696 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03937287-9e9c-44e5-8df1-cb6ae06b6856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:31.333984 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-03937287-9e9c-44e5-8df1-cb6ae06b6856 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:31.335446 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-03937287-9e9c-44e5-8df1-cb6ae06b6856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:31.347826 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:31.422472 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 166/338] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:31.679835 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55381ad4-e2ee-4e32-8517-8e519c9e25e8 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:31.692928 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032dfb4f-5f3a-4b75-8b10-40aeb1fe6284 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:31.693652 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032dfb4f-5f3a-4b75-8b10-40aeb1fe6284 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:31.694405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032dfb4f-5f3a-4b75-8b10-40aeb1fe6284 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:31.696436 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 167/339] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:31.701761 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e800e4-fd0e-48fe-bdf6-588710812006 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:31.702028 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e800e4-fd0e-48fe-bdf6-588710812006 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:31.702300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e800e4-fd0e-48fe-bdf6-588710812006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:31.734362 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-70e800e4-fd0e-48fe-bdf6-588710812006 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:31.735551 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-70e800e4-fd0e-48fe-bdf6-588710812006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:31.869897 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 168/340] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:31.871189 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfbd47a-7108-41d2-9121-8da2fa1b95a0 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:31.871564 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfbd47a-7108-41d2-9121-8da2fa1b95a0 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:31.871835 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfbd47a-7108-41d2-9121-8da2fa1b95a0 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:31.873109 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 169/341] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:31.882326 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361d178-28c6-4f17-91bd-50f2f1c1becf None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:31.882616 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361d178-28c6-4f17-91bd-50f2f1c1becf None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:31.882877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361d178-28c6-4f17-91bd-50f2f1c1becf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:31.883707 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55381ad4-e2ee-4e32-8517-8e519c9e25e8 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:31.919263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-4361d178-28c6-4f17-91bd-50f2f1c1becf None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:31.920596 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-4361d178-28c6-4f17-91bd-50f2f1c1becf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:31.951590 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70e013a-5581-48a4-9b3a-5e9b49439b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:31.952190 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e013a-5581-48a4-9b3a-5e9b49439b67 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:31.952464 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e013a-5581-48a4-9b3a-5e9b49439b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:31.952729 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e013a-5581-48a4-9b3a-5e9b49439b67 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:31.971333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70e013a-5581-48a4-9b3a-5e9b49439b67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:31.976060 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-d70e013a-5581-48a4-9b3a-5e9b49439b67 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:31.976060 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:31.976060 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:31.976060 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:31.976060 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:31.976060 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:31.976060 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:31.976060 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:31.976060 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:31.976060 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:31.976060 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:31.976060 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:31.976060 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:31.976060 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:31.976060 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:31.976060 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:31.977020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:31.977020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:31.977020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:31.977020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:31.977020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:31.977020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:31.977020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:31.977020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:31.977020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:31.977020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:31.977020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:31.977020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:31.977020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:31.977020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:31.977020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:31.977020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:31.977782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:31.977782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:31.977782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:31.977782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:31.977782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:31.977782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:31.977782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:31.977782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:31.977782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:31.977782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:31.977782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:31.977782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:31.980044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 173/342] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:31.981478 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70e013a-5581-48a4-9b3a-5e9b49439b67 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:32.041731 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:32.043392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 170/343] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 04:14:32.044494 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4361d178-28c6-4f17-91bd-50f2f1c1becf None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:32.059309 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:32.105795 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70e013a-5581-48a4-9b3a-5e9b49439b67 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:32.147334 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83076e3d-1c99-43b1-b245-28f4cd7c10e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:32.148273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83076e3d-1c99-43b1-b245-28f4cd7c10e4 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:32.148646 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83076e3d-1c99-43b1-b245-28f4cd7c10e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:32.148827 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83076e3d-1c99-43b1-b245-28f4cd7c10e4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:32.160785 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83076e3d-1c99-43b1-b245-28f4cd7c10e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:32.166405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-83076e3d-1c99-43b1-b245-28f4cd7c10e4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 04:14:32.166405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:32.166405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:32.166405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:32.166405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:32.166405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:32.166405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:32.166405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:32.166405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:32.166405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:32.166405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:32.166405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:32.166405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:32.166405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:32.166405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:32.166405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:32.167271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:32.167271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:32.167271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:32.167271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:32.167271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:32.167271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:32.167271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:32.167271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:32.167271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:32.167271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:32.167271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:32.167271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:32.167271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:32.167271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:32.167271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:32.167271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:32.168088 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:32.168088 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:32.168088 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:32.168088 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:32.168088 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:32.168088 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:32.168088 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:32.168088 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:32.168088 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:32.168088 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:32.168088 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 04:14:32.168088 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:32.172475 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 174/344] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Aug 23 04:14:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:32.174273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83076e3d-1c99-43b1-b245-28f4cd7c10e4 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:32.271270 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4361d178-28c6-4f17-91bd-50f2f1c1becf None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:32.284588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83076e3d-1c99-43b1-b245-28f4cd7c10e4 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:32.302999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:32.323093 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45133342-6dad-48e9-9325-eeffbb3c7c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:32.324045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45133342-6dad-48e9-9325-eeffbb3c7c3f None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:32.324259 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45133342-6dad-48e9-9325-eeffbb3c7c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:32.324534 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45133342-6dad-48e9-9325-eeffbb3c7c3f None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:32.325202 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45133342-6dad-48e9-9325-eeffbb3c7c3f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:32.325502 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45133342-6dad-48e9-9325-eeffbb3c7c3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:32.329269 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45133342-6dad-48e9-9325-eeffbb3c7c3f None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:32.341771 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 175/345] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:14:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:32.342723 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45133342-6dad-48e9-9325-eeffbb3c7c3f None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:32.375550 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dc9a2e-551a-43d0-b70d-c5359bae6011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:32.376739 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc9a2e-551a-43d0-b70d-c5359bae6011 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:32.377208 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc9a2e-551a-43d0-b70d-c5359bae6011 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:32.377659 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc9a2e-551a-43d0-b70d-c5359bae6011 None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:32.378731 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dc9a2e-551a-43d0-b70d-c5359bae6011 None admin] RBAC: Adding query filter params (name=member) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:32.379219 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dc9a2e-551a-43d0-b70d-c5359bae6011 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:32.380873 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dc9a2e-551a-43d0-b70d-c5359bae6011 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:32.399557 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 171/346] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Aug 23 04:14:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:32.400837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dc9a2e-551a-43d0-b70d-c5359bae6011 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:32.467940 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45133342-6dad-48e9-9325-eeffbb3c7c3f None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:32.499577 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d13b86-3c72-4c93-94b3-a75ada2ed071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:32.500548 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d13b86-3c72-4c93-94b3-a75ada2ed071 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:32.501092 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d13b86-3c72-4c93-94b3-a75ada2ed071 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:32.501366 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d13b86-3c72-4c93-94b3-a75ada2ed071 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:32.512872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:32.513313 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d13b86-3c72-4c93-94b3-a75ada2ed071 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:32.520353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-91d13b86-3c72-4c93-94b3-a75ada2ed071 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:32.520353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:32.520353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:32.520353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:32.520353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:32.520353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:32.520353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:32.520353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:32.520353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:32.520353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:32.520353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:32.520353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:32.520353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:32.520353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:32.520353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:32.520353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:32.521254 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:32.521254 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:32.521254 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:32.521254 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:32.521254 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:32.521254 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:32.521254 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:32.521254 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:32.521254 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:32.521254 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:32.521254 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:32.521254 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:32.521254 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:32.521254 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:32.521254 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:32.521254 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:32.522201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:32.522201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:32.522201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:32.522201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:32.522201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:32.522201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:32.522201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:32.522201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:32.522201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:32.522201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:32.522201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:32.522201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:32.522201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 176/347] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:32.522944 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d13b86-3c72-4c93-94b3-a75ada2ed071 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:32.559602 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dc9a2e-551a-43d0-b70d-c5359bae6011 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:32.566042 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:32.606684 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ea1fa-42af-4186-8b5e-34fad4bb18ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:32.607764 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ea1fa-42af-4186-8b5e-34fad4bb18ce None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:32.608237 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ea1fa-42af-4186-8b5e-34fad4bb18ce None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:32.608649 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ea1fa-42af-4186-8b5e-34fad4bb18ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:32.644220 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d13b86-3c72-4c93-94b3-a75ada2ed071 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:32.688069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35ea1fa-42af-4186-8b5e-34fad4bb18ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:32.689821 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35ea1fa-42af-4186-8b5e-34fad4bb18ce None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:32.693211 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819fc5b-483c-4050-9248-28f27a185f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:32.694801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819fc5b-483c-4050-9248-28f27a185f4c None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:32.695146 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819fc5b-483c-4050-9248-28f27a185f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:32.695427 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819fc5b-483c-4050-9248-28f27a185f4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:32.773834 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:32.777846 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f819fc5b-483c-4050-9248-28f27a185f4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:32.779641 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f819fc5b-483c-4050-9248-28f27a185f4c None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:32.821188 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 172/348] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:14:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:14:32.822193 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ea1fa-42af-4186-8b5e-34fad4bb18ce None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:32.824794 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:32.902025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 177/349] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:14:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:14:32.903131 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819fc5b-483c-4050-9248-28f27a185f4c None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:32.948723 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ea1fa-42af-4186-8b5e-34fad4bb18ce None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:32.988928 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b98ad7-edcb-4bb7-9c5a-cff292b722db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:32.989732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b98ad7-edcb-4bb7-9c5a-cff292b722db None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:32.990122 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b98ad7-edcb-4bb7-9c5a-cff292b722db None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:32.990481 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b98ad7-edcb-4bb7-9c5a-cff292b722db None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:33.002799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b98ad7-edcb-4bb7-9c5a-cff292b722db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:33.008112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-59b98ad7-edcb-4bb7-9c5a-cff292b722db None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 04:14:33.008112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:33.008112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:33.008112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:33.008112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:33.008112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:33.008112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:33.008112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:33.008112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:33.008112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:33.008112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:33.008112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:33.008112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:33.008112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:33.008112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:33.008112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:33.009017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:33.009017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:33.009017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:33.009017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:33.009017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:33.009017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:33.009017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:33.009017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:33.009017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:33.009017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:33.009017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:33.009017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:33.009017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:33.009017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:33.009017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:33.009017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:33.009817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:33.009817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:33.009817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:33.009817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:33.009817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:33.009817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:33.009817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:33.009817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:33.009817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:33.009817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:33.009817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 04:14:33.009817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:33.012100 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 173/350] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:33.013961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf7404-7b38-46b6-a386-4cfe90b4d424 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:33.014245 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf7404-7b38-46b6-a386-4cfe90b4d424 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:33.014515 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf7404-7b38-46b6-a386-4cfe90b4d424 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:33.015944 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 174/351] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:33.018116 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf7404-7b38-46b6-a386-4cfe90b4d424 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:33.044620 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819fc5b-483c-4050-9248-28f27a185f4c None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:33.057738 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:33.087703 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a36d1cc-d3c7-4e23-92dc-ccbcfcf90f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:33.088353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36d1cc-d3c7-4e23-92dc-ccbcfcf90f5b None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:33.088993 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36d1cc-d3c7-4e23-92dc-ccbcfcf90f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:33.089265 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36d1cc-d3c7-4e23-92dc-ccbcfcf90f5b None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:33.089939 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a36d1cc-d3c7-4e23-92dc-ccbcfcf90f5b None admin] RBAC: Adding query filter params (name=member) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:33.090243 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a36d1cc-d3c7-4e23-92dc-ccbcfcf90f5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:33.091678 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a36d1cc-d3c7-4e23-92dc-ccbcfcf90f5b None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:33.094280 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:33.102171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 178/352] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Aug 23 04:14:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:33.103374 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a36d1cc-d3c7-4e23-92dc-ccbcfcf90f5b None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:33.156386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf7404-7b38-46b6-a386-4cfe90b4d424 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:33.197887 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5570c7-92d1-49aa-bf0d-c389eb3edaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:33.198895 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5570c7-92d1-49aa-bf0d-c389eb3edaa3 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:33.199361 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5570c7-92d1-49aa-bf0d-c389eb3edaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:33.199841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5570c7-92d1-49aa-bf0d-c389eb3edaa3 None admin] PATH_INFO: `/v3/projects/e3148f82c47e4702b119cf867f5dcdcc` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:33.201144 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5570c7-92d1-49aa-bf0d-c389eb3edaa3 None admin] RBAC: Authorizing `identity:get_project(project_id=e3148f82c47e4702b119cf867f5dcdcc)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:33.202722 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5570c7-92d1-49aa-bf0d-c389eb3edaa3 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:33.204804 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 175/353] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:33 2022] GET /identity/v3/projects/e3148f82c47e4702b119cf867f5dcdcc => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:33.206042 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4984f68-cdef-4c5a-9dc9-3469ed3bd7c4 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:33.206475 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4984f68-cdef-4c5a-9dc9-3469ed3bd7c4 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:33.206878 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4984f68-cdef-4c5a-9dc9-3469ed3bd7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:33.228504 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a36d1cc-d3c7-4e23-92dc-ccbcfcf90f5b None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:33.230046 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-a4984f68-cdef-4c5a-9dc9-3469ed3bd7c4 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:33.231231 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a4984f68-cdef-4c5a-9dc9-3469ed3bd7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:33.276954 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba2c20-ec6e-4864-9014-93fa4c2d3fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:33.277601 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba2c20-ec6e-4864-9014-93fa4c2d3fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:33.277880 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba2c20-ec6e-4864-9014-93fa4c2d3fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:33.278122 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba2c20-ec6e-4864-9014-93fa4c2d3fd0 None admin] PATH_INFO: `/v3/projects/e3148f82c47e4702b119cf867f5dcdcc` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:33.279317 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba2c20-ec6e-4864-9014-93fa4c2d3fd0 None admin] RBAC: Authorizing `identity:get_project(project_id=e3148f82c47e4702b119cf867f5dcdcc)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:33.280797 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba2c20-ec6e-4864-9014-93fa4c2d3fd0 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:33.282553 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 179/354] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:33 2022] GET /identity/v3/projects/e3148f82c47e4702b119cf867f5dcdcc => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:33.283643 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba2c20-ec6e-4864-9014-93fa4c2d3fd0 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:33.325131 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 176/355] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:33.326519 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4984f68-cdef-4c5a-9dc9-3469ed3bd7c4 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:33.362415 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:33.366611 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:33.408919 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba2c20-ec6e-4864-9014-93fa4c2d3fd0 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:33.455575 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26826f-8889-4584-b23a-01a5be92bb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:33.456251 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26826f-8889-4584-b23a-01a5be92bb6c None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:33.456527 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26826f-8889-4584-b23a-01a5be92bb6c None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:33.456795 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26826f-8889-4584-b23a-01a5be92bb6c None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:33.457311 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4984f68-cdef-4c5a-9dc9-3469ed3bd7c4 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:33.457476 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26826f-8889-4584-b23a-01a5be92bb6c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:33.457761 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26826f-8889-4584-b23a-01a5be92bb6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:33.462471 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26826f-8889-4584-b23a-01a5be92bb6c None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:33.476089 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 180/356] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Aug 23 04:14:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:33.476089 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26826f-8889-4584-b23a-01a5be92bb6c None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:33.507491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67ff5d-c3d8-4810-9b84-fe41ee5bf84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:33.508813 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67ff5d-c3d8-4810-9b84-fe41ee5bf84b None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:33.508813 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67ff5d-c3d8-4810-9b84-fe41ee5bf84b None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:33.508813 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67ff5d-c3d8-4810-9b84-fe41ee5bf84b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:33.593549 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67ff5d-c3d8-4810-9b84-fe41ee5bf84b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:33.596214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67ff5d-c3d8-4810-9b84-fe41ee5bf84b None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:33.606997 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26826f-8889-4584-b23a-01a5be92bb6c None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:33.634480 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:33.647647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:33.652927 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025775b1-f649-4e34-8b0d-f62b9eb31f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:33.653743 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025775b1-f649-4e34-8b0d-f62b9eb31f8c None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:33.654024 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025775b1-f649-4e34-8b0d-f62b9eb31f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:33.654298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025775b1-f649-4e34-8b0d-f62b9eb31f8c None admin] PATH_INFO: `/v3/groups/7098ad8275714c318bdc7126575c8ceb` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:33.662144 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025775b1-f649-4e34-8b0d-f62b9eb31f8c None admin] RBAC: Authorizing `identity:get_group(group_id=7098ad8275714c318bdc7126575c8ceb)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:33.663691 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025775b1-f649-4e34-8b0d-f62b9eb31f8c None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:33.665416 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 181/357] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Aug 23 04:14:33 2022] GET /identity/v3/groups/7098ad8275714c318bdc7126575c8ceb => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:33.668760 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43821902-1ff0-43ea-9c3f-9f2ad06f87b4 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:33.668760 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43821902-1ff0-43ea-9c3f-9f2ad06f87b4 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:33.668760 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43821902-1ff0-43ea-9c3f-9f2ad06f87b4 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:33.669335 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 182/358] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:33.669885 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43821902-1ff0-43ea-9c3f-9f2ad06f87b4 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:33.723277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 177/359] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:14:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:14:33.724249 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67ff5d-c3d8-4810-9b84-fe41ee5bf84b None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:33.788710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43821902-1ff0-43ea-9c3f-9f2ad06f87b4 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:33.834154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061c5ef-fb8a-476c-8982-3515d37d6749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:33.834882 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061c5ef-fb8a-476c-8982-3515d37d6749 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:33.835223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061c5ef-fb8a-476c-8982-3515d37d6749 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:33.836357 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061c5ef-fb8a-476c-8982-3515d37d6749 None admin] PATH_INFO: `/v3/groups/7098ad8275714c318bdc7126575c8ceb` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:33.838319 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3061c5ef-fb8a-476c-8982-3515d37d6749 None admin] RBAC: Authorizing `identity:get_group(group_id=7098ad8275714c318bdc7126575c8ceb)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:33.840796 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3061c5ef-fb8a-476c-8982-3515d37d6749 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:33.843229 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 183/360] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Aug 23 04:14:33 2022] GET /identity/v3/groups/7098ad8275714c318bdc7126575c8ceb => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:33.843352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67ff5d-c3d8-4810-9b84-fe41ee5bf84b None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:33.844173 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061c5ef-fb8a-476c-8982-3515d37d6749 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:33.872990 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:33.882951 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f94a4-e589-4c84-a59e-994f3a92f646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:33.883607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f94a4-e589-4c84-a59e-994f3a92f646 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:33.883886 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f94a4-e589-4c84-a59e-994f3a92f646 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:33.886649 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f94a4-e589-4c84-a59e-994f3a92f646 None admin] PATH_INFO: `/v3/users/03e2250f56cc4006a687c702c65b78ea` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:33.887273 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f94a4-e589-4c84-a59e-994f3a92f646 None admin] RBAC: Authorizing `identity:get_user(user_id=03e2250f56cc4006a687c702c65b78ea)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:33.888800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f94a4-e589-4c84-a59e-994f3a92f646 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:33.890754 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 178/361] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:14:33 2022] GET /identity/v3/users/03e2250f56cc4006a687c702c65b78ea => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:33.891537 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f94a4-e589-4c84-a59e-994f3a92f646 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:33.914121 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:33.962360 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061c5ef-fb8a-476c-8982-3515d37d6749 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:34.009165 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d7608-7d34-4b61-8921-dd3e3aadf210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:34.009812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d7608-7d34-4b61-8921-dd3e3aadf210 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:34.010079 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d7608-7d34-4b61-8921-dd3e3aadf210 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:34.010345 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d7608-7d34-4b61-8921-dd3e3aadf210 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:34.010683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f94a4-e589-4c84-a59e-994f3a92f646 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:34.024536 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23d7608-7d34-4b61-8921-dd3e3aadf210 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:34.029185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-b23d7608-7d34-4b61-8921-dd3e3aadf210 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 04:14:34.029185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:34.029185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:34.029185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:34.029185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:34.029185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:34.029185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:34.029185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:34.029185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:34.029185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:34.029185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:34.029185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:34.029185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:34.029185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:34.029185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:34.029185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:34.030040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:34.030040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:34.030040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:34.030040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:34.030040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:34.030040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:34.030040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:34.030040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:34.030040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:34.030040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:34.030040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:34.030040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:34.030040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:34.030040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:34.030040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:34.030040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:34.030869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:34.030869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:34.030869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:34.030869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:34.030869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:34.030869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:34.030869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:34.030869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:34.030869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:34.030869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:34.030869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 04:14:34.030869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:34.030869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 184/362] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Aug 23 04:14:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:34.031835 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52259c-5ca3-4d35-b7b9-3432bd6da671 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:34.032070 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52259c-5ca3-4d35-b7b9-3432bd6da671 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:34.032222 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52259c-5ca3-4d35-b7b9-3432bd6da671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:34.051908 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174b5fe-afa2-4c0a-be17-2a467f586119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:34.052546 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174b5fe-afa2-4c0a-be17-2a467f586119 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:34.052872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174b5fe-afa2-4c0a-be17-2a467f586119 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:34.053138 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174b5fe-afa2-4c0a-be17-2a467f586119 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:34.053977 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8174b5fe-afa2-4c0a-be17-2a467f586119 None admin] RBAC: Adding query filter params (group.id=7098ad8275714c318bdc7126575c8ceb, role.id=4207a743edc04b0aba9197fe3554b2e3, scope.project.id=e3148f82c47e4702b119cf867f5dcdcc) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:34.054283 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8174b5fe-afa2-4c0a-be17-2a467f586119 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:34.054642 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-cd52259c-5ca3-4d35-b7b9-3432bd6da671 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:34.055860 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-cd52259c-5ca3-4d35-b7b9-3432bd6da671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:34.056876 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8174b5fe-afa2-4c0a-be17-2a467f586119 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:34.063459 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 179/363] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Aug 23 04:14:33 2022] GET /identity/v3/role_assignments?group.id=7098ad8275714c318bdc7126575c8ceb&scope.project.id=e3148f82c47e4702b119cf867f5dcdcc&role.id=4207a743edc04b0aba9197fe3554b2e3 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:34.064290 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174b5fe-afa2-4c0a-be17-2a467f586119 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:34.140779 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:34.151008 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 185/364] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:34.152477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52259c-5ca3-4d35-b7b9-3432bd6da671 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:34.209383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174b5fe-afa2-4c0a-be17-2a467f586119 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:34.225266 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:34.254202 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eae46b-a8b1-4c73-b422-ea86b3b38194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:34.256017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eae46b-a8b1-4c73-b422-ea86b3b38194 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:34.256265 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eae46b-a8b1-4c73-b422-ea86b3b38194 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:34.256543 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eae46b-a8b1-4c73-b422-ea86b3b38194 None admin] PATH_INFO: `/v3/projects/95a15538272e426ba99c0b50e2f30f52` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:34.264024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eae46b-a8b1-4c73-b422-ea86b3b38194 None admin] RBAC: Authorizing `identity:get_project(project_id=95a15538272e426ba99c0b50e2f30f52)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:34.265994 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eae46b-a8b1-4c73-b422-ea86b3b38194 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:34.268788 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 180/365] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:34 2022] GET /identity/v3/projects/95a15538272e426ba99c0b50e2f30f52 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:34.269823 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eae46b-a8b1-4c73-b422-ea86b3b38194 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:34.292443 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52259c-5ca3-4d35-b7b9-3432bd6da671 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:34.328882 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41c37b-b4f2-4f7b-8c21-55191aab9685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:34.329506 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41c37b-b4f2-4f7b-8c21-55191aab9685 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:34.329784 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41c37b-b4f2-4f7b-8c21-55191aab9685 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:34.330056 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41c37b-b4f2-4f7b-8c21-55191aab9685 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:34.330800 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41c37b-b4f2-4f7b-8c21-55191aab9685 None admin] RBAC: Adding query filter params (group.id=7098ad8275714c318bdc7126575c8ceb, role.id=aa03504b4fb642d288d07f4335e291e1, scope.project.id=e3148f82c47e4702b119cf867f5dcdcc) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:34.331109 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41c37b-b4f2-4f7b-8c21-55191aab9685 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:34.332659 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41c37b-b4f2-4f7b-8c21-55191aab9685 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:34.340288 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 186/366] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Aug 23 04:14:34 2022] GET /identity/v3/role_assignments?group.id=7098ad8275714c318bdc7126575c8ceb&scope.project.id=e3148f82c47e4702b119cf867f5dcdcc&role.id=aa03504b4fb642d288d07f4335e291e1 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:34.341236 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41c37b-b4f2-4f7b-8c21-55191aab9685 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:34.439324 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eae46b-a8b1-4c73-b422-ea86b3b38194 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:34.488297 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:34.493443 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d5d13-12cf-4067-b0d7-39187a6cb6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:34.494510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d5d13-12cf-4067-b0d7-39187a6cb6b3 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:34.496674 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d5d13-12cf-4067-b0d7-39187a6cb6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:34.496674 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d5d13-12cf-4067-b0d7-39187a6cb6b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:34.496674 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d5d13-12cf-4067-b0d7-39187a6cb6b3 None admin] RBAC: Adding query filter params (role.id=4be260163ba342e9b42232a49eec9fec, user.id=03e2250f56cc4006a687c702c65b78ea, scope.system=all) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:34.497085 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d5d13-12cf-4067-b0d7-39187a6cb6b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:34.498887 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d5d13-12cf-4067-b0d7-39187a6cb6b3 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:34.513329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41c37b-b4f2-4f7b-8c21-55191aab9685 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:34.518175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 181/367] 10.222.0.18 () {56 vars in 1308 bytes} [Tue Aug 23 04:14:34 2022] GET /identity/v3/role_assignments?user.id=03e2250f56cc4006a687c702c65b78ea&scope.system=all&role.id=4be260163ba342e9b42232a49eec9fec => generated 510 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:34.519187 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d5d13-12cf-4067-b0d7-39187a6cb6b3 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:34.535823 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:34.573917 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2299f60-c775-42c5-b8b0-bbbbfb60036c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:34.574973 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2299f60-c775-42c5-b8b0-bbbbfb60036c None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:34.575446 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2299f60-c775-42c5-b8b0-bbbbfb60036c None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:34.575941 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2299f60-c775-42c5-b8b0-bbbbfb60036c None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:34.576829 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2299f60-c775-42c5-b8b0-bbbbfb60036c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:34.577325 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2299f60-c775-42c5-b8b0-bbbbfb60036c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:34.579590 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2299f60-c775-42c5-b8b0-bbbbfb60036c None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:34.592766 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 187/368] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:14:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:34.593975 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2299f60-c775-42c5-b8b0-bbbbfb60036c None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:34.669097 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d5d13-12cf-4067-b0d7-39187a6cb6b3 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:34.716466 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd3a048-336b-4e1c-a3a6-fec2e80f8717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:34.717553 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3a048-336b-4e1c-a3a6-fec2e80f8717 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:34.718024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3a048-336b-4e1c-a3a6-fec2e80f8717 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:34.718485 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3a048-336b-4e1c-a3a6-fec2e80f8717 None admin] PATH_INFO: `/v3/groups/7098ad8275714c318bdc7126575c8ceb` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:34.720031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd3a048-336b-4e1c-a3a6-fec2e80f8717 None admin] RBAC: Authorizing `identity:get_group(group_id=7098ad8275714c318bdc7126575c8ceb)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:34.721697 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd3a048-336b-4e1c-a3a6-fec2e80f8717 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:34.725432 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 182/369] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Aug 23 04:14:34 2022] GET /identity/v3/groups/7098ad8275714c318bdc7126575c8ceb => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:34.726666 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd3a048-336b-4e1c-a3a6-fec2e80f8717 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:34.743548 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2299f60-c775-42c5-b8b0-bbbbfb60036c None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:34.783337 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908fdfca-6035-4da3-bf0f-da1e3614b24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:34.783999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fdfca-6035-4da3-bf0f-da1e3614b24a None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:34.784278 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fdfca-6035-4da3-bf0f-da1e3614b24a None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:34.784551 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fdfca-6035-4da3-bf0f-da1e3614b24a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:34.789387 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:34.796992 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908fdfca-6035-4da3-bf0f-da1e3614b24a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:34.802926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-908fdfca-6035-4da3-bf0f-da1e3614b24a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:14:34.802926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:34.802926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:34.802926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:34.802926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:34.802926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:34.802926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:34.802926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:34.802926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:34.802926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:34.802926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:34.802926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:34.802926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:34.802926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:34.802926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:34.802926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:34.803868 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:34.803868 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:34.803868 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:34.803868 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:34.803868 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:34.803868 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:34.803868 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:34.803868 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:34.803868 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:34.803868 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:34.803868 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:34.803868 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:34.803868 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:34.803868 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:34.803868 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:34.803868 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:34.804693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:34.804693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:34.804693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:34.804693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:34.804693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:34.804693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:34.804693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:34.804693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:34.804693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:34.804693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:34.804693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:14:34.804693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:34.808157 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 188/370] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:14:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:34.809501 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908fdfca-6035-4da3-bf0f-da1e3614b24a None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:34.845739 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd3a048-336b-4e1c-a3a6-fec2e80f8717 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:34.848066 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:34.883516 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdfbcee-6f67-4473-8525-aa7a60fa2df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:34.884537 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdfbcee-6f67-4473-8525-aa7a60fa2df9 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:34.885018 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdfbcee-6f67-4473-8525-aa7a60fa2df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:34.885470 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdfbcee-6f67-4473-8525-aa7a60fa2df9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:34.918472 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908fdfca-6035-4da3-bf0f-da1e3614b24a None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:34.960016 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189bde2-250e-45a8-bcea-03e0395b7d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:34.960726 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189bde2-250e-45a8-bcea-03e0395b7d9b None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:34.961008 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189bde2-250e-45a8-bcea-03e0395b7d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:34.961285 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189bde2-250e-45a8-bcea-03e0395b7d9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:34.961725 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdfbcee-6f67-4473-8525-aa7a60fa2df9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:34.962036 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189bde2-250e-45a8-bcea-03e0395b7d9b None admin] RBAC: Adding query filter params (group.id=7098ad8275714c318bdc7126575c8ceb, role.id=aa03504b4fb642d288d07f4335e291e1, scope.project.id=95a15538272e426ba99c0b50e2f30f52) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:34.962342 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189bde2-250e-45a8-bcea-03e0395b7d9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:34.964206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189bde2-250e-45a8-bcea-03e0395b7d9b None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:34.964484 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdfbcee-6f67-4473-8525-aa7a60fa2df9 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:34.976720 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 189/371] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Aug 23 04:14:34 2022] GET /identity/v3/role_assignments?group.id=7098ad8275714c318bdc7126575c8ceb&scope.project.id=95a15538272e426ba99c0b50e2f30f52&role.id=aa03504b4fb642d288d07f4335e291e1 => generated 260 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:34.977835 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189bde2-250e-45a8-bcea-03e0395b7d9b None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:35.064437 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:35.119020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:35.123851 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189bde2-250e-45a8-bcea-03e0395b7d9b None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:35.129774 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 183/372] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:14:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:14:35.135819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdfbcee-6f67-4473-8525-aa7a60fa2df9 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:35.169515 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b4d8a-d768-4432-886e-02be4e693d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:35.170507 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b4d8a-d768-4432-886e-02be4e693d45 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:35.170855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b4d8a-d768-4432-886e-02be4e693d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:35.171140 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b4d8a-d768-4432-886e-02be4e693d45 None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:35.172137 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5b4d8a-d768-4432-886e-02be4e693d45 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:35.172450 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5b4d8a-d768-4432-886e-02be4e693d45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:35.175202 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5b4d8a-d768-4432-886e-02be4e693d45 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:35.190392 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 190/373] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:35.192856 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b4d8a-d768-4432-886e-02be4e693d45 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:35.278903 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdfbcee-6f67-4473-8525-aa7a60fa2df9 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:35.320329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b4d8a-d768-4432-886e-02be4e693d45 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:35.325855 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab513214-3ba9-4a77-b42a-871780add4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:35.326689 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab513214-3ba9-4a77-b42a-871780add4a2 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:35.327103 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab513214-3ba9-4a77-b42a-871780add4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:35.327527 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab513214-3ba9-4a77-b42a-871780add4a2 None admin] PATH_INFO: `/v3/projects/95a15538272e426ba99c0b50e2f30f52` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:35.328846 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab513214-3ba9-4a77-b42a-871780add4a2 None admin] RBAC: Authorizing `identity:get_project(project_id=95a15538272e426ba99c0b50e2f30f52)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:35.331084 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab513214-3ba9-4a77-b42a-871780add4a2 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:35.333759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 184/374] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:35 2022] GET /identity/v3/projects/95a15538272e426ba99c0b50e2f30f52 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:14:35.340319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab513214-3ba9-4a77-b42a-871780add4a2 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:35.368236 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b10e83-2fe2-44da-825c-eb774366716d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:35.368993 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b10e83-2fe2-44da-825c-eb774366716d None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:35.369383 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b10e83-2fe2-44da-825c-eb774366716d None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:35.369597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:35.369826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b10e83-2fe2-44da-825c-eb774366716d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:35.410617 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:35.460861 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b10e83-2fe2-44da-825c-eb774366716d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:35.460861 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b10e83-2fe2-44da-825c-eb774366716d None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:35.496341 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab513214-3ba9-4a77-b42a-871780add4a2 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:35.542774 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae2d31-7c37-44c9-b34e-5f3f6aa725fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:35.543809 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae2d31-7c37-44c9-b34e-5f3f6aa725fa None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:35.544374 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae2d31-7c37-44c9-b34e-5f3f6aa725fa None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:35.544826 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae2d31-7c37-44c9-b34e-5f3f6aa725fa None admin] PATH_INFO: `/v3/groups/7098ad8275714c318bdc7126575c8ceb` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:35.546114 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae2d31-7c37-44c9-b34e-5f3f6aa725fa None admin] RBAC: Authorizing `identity:get_group(group_id=7098ad8275714c318bdc7126575c8ceb)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:35.547851 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae2d31-7c37-44c9-b34e-5f3f6aa725fa None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:35.551872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 185/375] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Aug 23 04:14:35 2022] GET /identity/v3/groups/7098ad8275714c318bdc7126575c8ceb => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:35.558500 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae2d31-7c37-44c9-b34e-5f3f6aa725fa None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:35.582023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 191/376] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:14:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 04:14:35.587424 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b10e83-2fe2-44da-825c-eb774366716d None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:35.656247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:35.687284 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:35.699658 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b10e83-2fe2-44da-825c-eb774366716d None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:35.704111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae2d31-7c37-44c9-b34e-5f3f6aa725fa None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:35.743439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903edf76-023a-4f49-97b9-8602626d0ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:35.744671 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903edf76-023a-4f49-97b9-8602626d0ab5 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:35.744946 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903edf76-023a-4f49-97b9-8602626d0ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:35.745213 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903edf76-023a-4f49-97b9-8602626d0ab5 None admin] PATH_INFO: `/v3/projects/ba69f2ba4f0d496e9c7607a800699841` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:35.747871 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903edf76-023a-4f49-97b9-8602626d0ab5 None admin] RBAC: Authorizing `identity:get_project(project_id=ba69f2ba4f0d496e9c7607a800699841)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:35.749347 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903edf76-023a-4f49-97b9-8602626d0ab5 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:35.753839 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16728c60-dc20-4fb2-b26e-9182f60d41a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:35.754452 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16728c60-dc20-4fb2-b26e-9182f60d41a7 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:35.754724 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16728c60-dc20-4fb2-b26e-9182f60d41a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:35.754989 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16728c60-dc20-4fb2-b26e-9182f60d41a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:35.755821 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16728c60-dc20-4fb2-b26e-9182f60d41a7 None admin] RBAC: Adding query filter params (group.id=7098ad8275714c318bdc7126575c8ceb, role.id=4207a743edc04b0aba9197fe3554b2e3, scope.project.id=95a15538272e426ba99c0b50e2f30f52) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:35.756152 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 192/377] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:35 2022] GET /identity/v3/projects/ba69f2ba4f0d496e9c7607a800699841 => generated 344 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:35.756352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16728c60-dc20-4fb2-b26e-9182f60d41a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:35.758147 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16728c60-dc20-4fb2-b26e-9182f60d41a7 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:35.758790 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903edf76-023a-4f49-97b9-8602626d0ab5 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:35.765843 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 186/378] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Aug 23 04:14:35 2022] GET /identity/v3/role_assignments?group.id=7098ad8275714c318bdc7126575c8ceb&scope.project.id=95a15538272e426ba99c0b50e2f30f52&role.id=4207a743edc04b0aba9197fe3554b2e3 => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:35.897618 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903edf76-023a-4f49-97b9-8602626d0ab5 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:35.929525 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:35.935784 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964bc83-2b71-4f8a-aef9-15f2b0d267ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:35.936591 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964bc83-2b71-4f8a-aef9-15f2b0d267ad None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:35.937268 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964bc83-2b71-4f8a-aef9-15f2b0d267ad None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:35.937730 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964bc83-2b71-4f8a-aef9-15f2b0d267ad None admin] PATH_INFO: `/v3/groups/8c7014f78086414a80a7008cf794a331` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:35.943431 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964bc83-2b71-4f8a-aef9-15f2b0d267ad None admin] RBAC: Authorizing `identity:get_group(group_id=8c7014f78086414a80a7008cf794a331)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:35.945106 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964bc83-2b71-4f8a-aef9-15f2b0d267ad None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:35.947102 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 193/379] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Aug 23 04:14:35 2022] GET /identity/v3/groups/8c7014f78086414a80a7008cf794a331 => generated 229 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:35.952754 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16728c60-dc20-4fb2-b26e-9182f60d41a7 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:36.096348 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16728c60-dc20-4fb2-b26e-9182f60d41a7 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:36.141872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ad7c3-43c7-4813-83a1-0f632ca37cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:36.142829 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ad7c3-43c7-4813-83a1-0f632ca37cdf None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:36.143243 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ad7c3-43c7-4813-83a1-0f632ca37cdf None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:36.143644 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ad7c3-43c7-4813-83a1-0f632ca37cdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:36.144554 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8ad7c3-43c7-4813-83a1-0f632ca37cdf None admin] RBAC: Adding query filter params (group.id=8c7014f78086414a80a7008cf794a331, role.id=52b31061b72c45ccb76dd26bf31f8d3f, scope.project.id=ba69f2ba4f0d496e9c7607a800699841) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:36.144994 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8ad7c3-43c7-4813-83a1-0f632ca37cdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:36.146577 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8ad7c3-43c7-4813-83a1-0f632ca37cdf None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:36.152853 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 187/380] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Aug 23 04:14:35 2022] GET /identity/v3/role_assignments?group.id=8c7014f78086414a80a7008cf794a331&scope.project.id=ba69f2ba4f0d496e9c7607a800699841&role.id=52b31061b72c45ccb76dd26bf31f8d3f => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:36.672172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0bd2a1-a112-4a38-a3cf-e8d4127d19ec None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:36.672686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0bd2a1-a112-4a38-a3cf-e8d4127d19ec None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:36.672686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0bd2a1-a112-4a38-a3cf-e8d4127d19ec None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:36.674351 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 194/381] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:36.685339 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1dea6f-4699-4ee2-9c3b-45e373767b2d None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:36.685747 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1dea6f-4699-4ee2-9c3b-45e373767b2d None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:36.685927 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1dea6f-4699-4ee2-9c3b-45e373767b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:36.710469 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-cb1dea6f-4699-4ee2-9c3b-45e373767b2d None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:36.711584 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-cb1dea6f-4699-4ee2-9c3b-45e373767b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:36.729292 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:36.802218 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 188/382] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:36.853552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a929aaf-34e7-4d87-ae47-cb27a31f2494 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:36.853866 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a929aaf-34e7-4d87-ae47-cb27a31f2494 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:36.854107 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a929aaf-34e7-4d87-ae47-cb27a31f2494 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:36.855576 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 195/383] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:36.860982 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7fb9dd-2974-4248-9a0f-e406fea5a42c None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:36.861269 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7fb9dd-2974-4248-9a0f-e406fea5a42c None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:36.861541 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7fb9dd-2974-4248-9a0f-e406fea5a42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:36.884141 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-9b7fb9dd-2974-4248-9a0f-e406fea5a42c None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:36.885343 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-9b7fb9dd-2974-4248-9a0f-e406fea5a42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:36.957213 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 189/384] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:37.197861 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a929aaf-34e7-4d87-ae47-cb27a31f2494 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:37.279510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7fb9dd-2974-4248-9a0f-e406fea5a42c None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:37.310164 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a929aaf-34e7-4d87-ae47-cb27a31f2494 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:37.349440 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59904476-ca42-474e-958c-d58b804272ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:37.350089 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59904476-ca42-474e-958c-d58b804272ce None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:37.350366 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59904476-ca42-474e-958c-d58b804272ce None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:37.350769 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59904476-ca42-474e-958c-d58b804272ce None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:37.373591 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59904476-ca42-474e-958c-d58b804272ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:37.380904 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-59904476-ca42-474e-958c-d58b804272ce None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 04:14:37.380904 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:37.380904 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:37.380904 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:37.380904 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:37.380904 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:37.380904 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:37.380904 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:37.380904 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:37.380904 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:37.380904 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:37.380904 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:37.380904 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:37.380904 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:37.380904 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:37.380904 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:37.382046 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:37.382046 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:37.382046 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:37.382046 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:37.382046 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:37.382046 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:37.382046 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:37.382046 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:37.382046 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:37.382046 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:37.382046 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:37.382046 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:37.382046 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:37.382046 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:37.382046 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:37.382046 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:37.382873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:37.382873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:37.382873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:37.382873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:37.382873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:37.382873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:37.382873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:37.382873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:37.382873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:37.382873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:37.382873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 04:14:37.382873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:37.385484 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 196/385] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Aug 23 04:14:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:37.388708 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59904476-ca42-474e-958c-d58b804272ce None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:37.426510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:37.467498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7fb9dd-2974-4248-9a0f-e406fea5a42c None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:37.500828 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:37.547137 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b0f1a-d4b4-43d5-bdfc-25091a856235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:37.548017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b0f1a-d4b4-43d5-bdfc-25091a856235 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:37.548397 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b0f1a-d4b4-43d5-bdfc-25091a856235 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:37.548766 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b0f1a-d4b4-43d5-bdfc-25091a856235 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:37.560245 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b0f1a-d4b4-43d5-bdfc-25091a856235 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:37.565138 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-1d2b0f1a-d4b4-43d5-bdfc-25091a856235 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:37.565138 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:37.565138 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:37.565138 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:37.565138 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:37.565138 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:37.565138 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:37.565138 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:37.565138 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:37.565138 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:37.565138 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:37.565138 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:37.565138 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:37.565138 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:37.565138 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:37.565138 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:37.566210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:37.566210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:37.566210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:37.566210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:37.566210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:37.566210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:37.566210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:37.566210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:37.566210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:37.566210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:37.566210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:37.566210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:37.566210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:37.566210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:37.566210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:37.566210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:37.567142 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:37.567142 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:37.567142 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:37.567142 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:37.567142 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:37.567142 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:37.567142 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:37.567142 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:37.567142 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:37.567142 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:37.567142 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:37.567142 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:37.569065 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 190/386] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:37.576854 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b0f1a-d4b4-43d5-bdfc-25091a856235 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:37.607960 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59904476-ca42-474e-958c-d58b804272ce None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:37.671637 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed4dc3-7658-4de0-83fc-bc0b275e5373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:37.672555 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed4dc3-7658-4de0-83fc-bc0b275e5373 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:37.673421 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed4dc3-7658-4de0-83fc-bc0b275e5373 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:37.673849 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed4dc3-7658-4de0-83fc-bc0b275e5373 None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:37.674670 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed4dc3-7658-4de0-83fc-bc0b275e5373 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:37.675120 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed4dc3-7658-4de0-83fc-bc0b275e5373 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:37.676661 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed4dc3-7658-4de0-83fc-bc0b275e5373 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:37.693381 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 197/387] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:14:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:37.697892 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed4dc3-7658-4de0-83fc-bc0b275e5373 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:37.739662 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b0f1a-d4b4-43d5-bdfc-25091a856235 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:37.782784 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef0661-b14e-422f-a31f-f85f49142ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:37.782784 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef0661-b14e-422f-a31f-f85f49142ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:37.782784 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef0661-b14e-422f-a31f-f85f49142ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:37.782784 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef0661-b14e-422f-a31f-f85f49142ac2 None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:37.783253 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef0661-b14e-422f-a31f-f85f49142ac2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:37.783661 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef0661-b14e-422f-a31f-f85f49142ac2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:37.785212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef0661-b14e-422f-a31f-f85f49142ac2 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:37.788244 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:37.802770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 191/388] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Aug 23 04:14:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:37.808474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef0661-b14e-422f-a31f-f85f49142ac2 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:37.862437 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed4dc3-7658-4de0-83fc-bc0b275e5373 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:37.879521 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:37.899540 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea7a67-804b-4709-8ba6-88da46b48c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:37.900223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea7a67-804b-4709-8ba6-88da46b48c25 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:37.900551 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea7a67-804b-4709-8ba6-88da46b48c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:37.900821 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea7a67-804b-4709-8ba6-88da46b48c25 None admin] PATH_INFO: `/v3/groups/7098ad8275714c318bdc7126575c8ceb` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:37.901928 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ea7a67-804b-4709-8ba6-88da46b48c25 None admin] RBAC: Authorizing `identity:get_group(group_id=7098ad8275714c318bdc7126575c8ceb)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:37.903387 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ea7a67-804b-4709-8ba6-88da46b48c25 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:37.905342 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 198/389] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Aug 23 04:14:37 2022] GET /identity/v3/groups/7098ad8275714c318bdc7126575c8ceb => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:37.906537 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6caf80-f5f7-421d-a976-4e20cf56336e None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:37.906818 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6caf80-f5f7-421d-a976-4e20cf56336e None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:37.907088 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6caf80-f5f7-421d-a976-4e20cf56336e None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:37.912570 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 199/390] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:37.913425 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6caf80-f5f7-421d-a976-4e20cf56336e None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:37.923479 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef0661-b14e-422f-a31f-f85f49142ac2 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:37.958121 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c6ef3-38aa-4d4b-b62b-76d26912a041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:37.958764 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c6ef3-38aa-4d4b-b62b-76d26912a041 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:37.959044 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c6ef3-38aa-4d4b-b62b-76d26912a041 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:37.959318 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c6ef3-38aa-4d4b-b62b-76d26912a041 None admin] PATH_INFO: `/v3/groups/7098ad8275714c318bdc7126575c8ceb` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:37.960730 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c6ef3-38aa-4d4b-b62b-76d26912a041 None admin] RBAC: Authorizing `identity:get_group(group_id=7098ad8275714c318bdc7126575c8ceb)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:37.962141 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c6ef3-38aa-4d4b-b62b-76d26912a041 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:37.964474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 192/391] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Aug 23 04:14:37 2022] GET /identity/v3/groups/7098ad8275714c318bdc7126575c8ceb => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:37.965635 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0468c-df83-457f-b987-75f926060dee None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:37.965917 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0468c-df83-457f-b987-75f926060dee None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:37.966191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0468c-df83-457f-b987-75f926060dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:37.984052 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-07f0468c-df83-457f-b987-75f926060dee None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:37.985274 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-07f0468c-df83-457f-b987-75f926060dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:38.023091 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6caf80-f5f7-421d-a976-4e20cf56336e None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:38.048293 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:38.055147 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 193/392] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:38.057046 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0468c-df83-457f-b987-75f926060dee None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:38.067886 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdceb33-94c0-4ff7-bbba-6088d5114500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:38.068609 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdceb33-94c0-4ff7-bbba-6088d5114500 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:38.068883 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdceb33-94c0-4ff7-bbba-6088d5114500 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:38.069241 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdceb33-94c0-4ff7-bbba-6088d5114500 None admin] PATH_INFO: `/v3/projects/e3148f82c47e4702b119cf867f5dcdcc` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:38.070546 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdceb33-94c0-4ff7-bbba-6088d5114500 None admin] RBAC: Authorizing `identity:get_project(project_id=e3148f82c47e4702b119cf867f5dcdcc)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:38.074071 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdceb33-94c0-4ff7-bbba-6088d5114500 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:38.078792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 200/393] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:37 2022] GET /identity/v3/projects/e3148f82c47e4702b119cf867f5dcdcc => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:38.084524 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdceb33-94c0-4ff7-bbba-6088d5114500 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:38.189691 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:38.203837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0468c-df83-457f-b987-75f926060dee None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:38.240396 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdceb33-94c0-4ff7-bbba-6088d5114500 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:38.240613 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c67a4fc-2be3-47e3-91ee-98376e16026f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:38.241302 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67a4fc-2be3-47e3-91ee-98376e16026f None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:38.241586 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67a4fc-2be3-47e3-91ee-98376e16026f None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:38.241861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67a4fc-2be3-47e3-91ee-98376e16026f None admin] PATH_INFO: `/v3/projects/e3148f82c47e4702b119cf867f5dcdcc` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:38.243006 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c67a4fc-2be3-47e3-91ee-98376e16026f None admin] RBAC: Authorizing `identity:get_project(project_id=e3148f82c47e4702b119cf867f5dcdcc)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:38.244819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c67a4fc-2be3-47e3-91ee-98376e16026f None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:38.246120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 194/394] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:38 2022] GET /identity/v3/projects/e3148f82c47e4702b119cf867f5dcdcc => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:38.250832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1377c-d07a-41ea-b44e-d0e07cbf2cbd None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:38.250832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1377c-d07a-41ea-b44e-d0e07cbf2cbd None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:38.250832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1377c-d07a-41ea-b44e-d0e07cbf2cbd None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:38.250832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 195/395] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:38.254045 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1377c-d07a-41ea-b44e-d0e07cbf2cbd None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:38.283883 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e752b8-ff79-4101-9227-f9e84808d307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:38.285111 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e752b8-ff79-4101-9227-f9e84808d307 None admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:38.285687 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e752b8-ff79-4101-9227-f9e84808d307 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:38.286150 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e752b8-ff79-4101-9227-f9e84808d307 None admin] PATH_INFO: `/v3/projects/e3148f82c47e4702b119cf867f5dcdcc/groups/7098ad8275714c318bdc7126575c8ceb/roles/4207a743edc04b0aba9197fe3554b2e3` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:38.290552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e752b8-ff79-4101-9227-f9e84808d307 None admin] RBAC: Authorizing `identity:create_grant(project_id=e3148f82c47e4702b119cf867f5dcdcc, group_id=7098ad8275714c318bdc7126575c8ceb, role_id=4207a743edc04b0aba9197fe3554b2e3)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:38.292460 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e752b8-ff79-4101-9227-f9e84808d307 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:38.308704 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 201/396] 10.222.0.18 () {58 vars in 1442 bytes} [Tue Aug 23 04:14:38 2022] PUT /identity/v3/projects/e3148f82c47e4702b119cf867f5dcdcc/groups/7098ad8275714c318bdc7126575c8ceb/roles/4207a743edc04b0aba9197fe3554b2e3 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:14:38.310638 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958c20e-0d22-4c2f-9cd3-7d6b8994be5c None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:38.311184 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958c20e-0d22-4c2f-9cd3-7d6b8994be5c None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:38.311775 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958c20e-0d22-4c2f-9cd3-7d6b8994be5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:38.325203 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:38.334224 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-5958c20e-0d22-4c2f-9cd3-7d6b8994be5c None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:38.337025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-5958c20e-0d22-4c2f-9cd3-7d6b8994be5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:38.384670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1377c-d07a-41ea-b44e-d0e07cbf2cbd None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:38.418501 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 202/397] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:38.419257 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5958c20e-0d22-4c2f-9cd3-7d6b8994be5c None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:38.434342 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a0318-f0b9-452e-b880-51e6bfd33225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:38.435546 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a0318-f0b9-452e-b880-51e6bfd33225 None admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:38.436082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a0318-f0b9-452e-b880-51e6bfd33225 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:38.436585 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a0318-f0b9-452e-b880-51e6bfd33225 None admin] PATH_INFO: `/v3/projects/e3148f82c47e4702b119cf867f5dcdcc/groups/7098ad8275714c318bdc7126575c8ceb/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:38.439131 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a0318-f0b9-452e-b880-51e6bfd33225 None admin] RBAC: Authorizing `identity:create_grant(project_id=e3148f82c47e4702b119cf867f5dcdcc, group_id=7098ad8275714c318bdc7126575c8ceb, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:38.440930 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a0318-f0b9-452e-b880-51e6bfd33225 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:38.463945 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 196/398] 10.222.0.18 () {58 vars in 1442 bytes} [Tue Aug 23 04:14:38 2022] PUT /identity/v3/projects/e3148f82c47e4702b119cf867f5dcdcc/groups/7098ad8275714c318bdc7126575c8ceb/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:14:38.463945 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394628c-8516-4973-afa6-db6f0c632868 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:38.463945 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394628c-8516-4973-afa6-db6f0c632868 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:38.463945 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394628c-8516-4973-afa6-db6f0c632868 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:38.464944 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 197/399] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:38.468946 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b07e03-58f7-48d4-a849-68a16800e11f None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:38.468946 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b07e03-58f7-48d4-a849-68a16800e11f None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:38.468946 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b07e03-58f7-48d4-a849-68a16800e11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:38.494348 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-12b07e03-58f7-48d4-a849-68a16800e11f None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:38.496048 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-12b07e03-58f7-48d4-a849-68a16800e11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:38.509930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:38.580279 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5958c20e-0d22-4c2f-9cd3-7d6b8994be5c None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:38.590937 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 198/400] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:38.620494 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2dfb3-464a-4dfe-aed8-36da96c70a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:38.621558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2dfb3-464a-4dfe-aed8-36da96c70a07 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:38.622039 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2dfb3-464a-4dfe-aed8-36da96c70a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:38.622508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2dfb3-464a-4dfe-aed8-36da96c70a07 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:38.633590 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2dfb3-464a-4dfe-aed8-36da96c70a07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:38.638623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-bfd2dfb3-464a-4dfe-aed8-36da96c70a07 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:38.638623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:38.638623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:38.638623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:38.638623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:38.638623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:38.638623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:38.638623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:38.638623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:38.638623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:38.638623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:38.638623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:38.638623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:38.638623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:38.638623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:38.638623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:38.639453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:38.639453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:38.639453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:38.639453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:38.639453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:38.639453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:38.639453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:38.639453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:38.639453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:38.639453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:38.639453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:38.639453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:38.639453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:38.639453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:38.639453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:38.639453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:38.640274 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:38.640274 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:38.640274 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:38.640274 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:38.640274 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:38.640274 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:38.640274 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:38.640274 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:38.640274 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:38.640274 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:38.640274 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:38.640274 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:38.642505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 203/401] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:38.648459 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b07e03-58f7-48d4-a849-68a16800e11f None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:38.721500 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2dfb3-464a-4dfe-aed8-36da96c70a07 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:38.732757 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:38.809284 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b07e03-58f7-48d4-a849-68a16800e11f None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:38.871786 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2dfb3-464a-4dfe-aed8-36da96c70a07 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:38.876236 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:38.884512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c362c8-034f-4826-80e9-a836e2863a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:38.885454 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c362c8-034f-4826-80e9-a836e2863a0c None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:38.885951 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c362c8-034f-4826-80e9-a836e2863a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:38.886415 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c362c8-034f-4826-80e9-a836e2863a0c None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:38.887295 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c362c8-034f-4826-80e9-a836e2863a0c None admin] RBAC: Adding query filter params (name=member) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:38.887807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c362c8-034f-4826-80e9-a836e2863a0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:38.889428 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c362c8-034f-4826-80e9-a836e2863a0c None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:38.905338 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 199/402] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Aug 23 04:14:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:38.915708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c362c8-034f-4826-80e9-a836e2863a0c None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:38.930988 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877b858-b3cd-4c24-aefe-34f02b78be81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:38.931633 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877b858-b3cd-4c24-aefe-34f02b78be81 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:38.931930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877b858-b3cd-4c24-aefe-34f02b78be81 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:38.932240 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877b858-b3cd-4c24-aefe-34f02b78be81 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:38.944874 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5877b858-b3cd-4c24-aefe-34f02b78be81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:38.950452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-5877b858-b3cd-4c24-aefe-34f02b78be81 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 04:14:38.950452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:38.950452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:38.950452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:38.950452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:38.950452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:38.950452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:38.950452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:38.950452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:38.950452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:38.950452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:38.950452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:38.950452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:38.950452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:38.950452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:38.950452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:38.951308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:38.951308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:38.951308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:38.951308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:38.951308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:38.951308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:38.951308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:38.951308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:38.951308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:38.951308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:38.951308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:38.951308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:38.951308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:38.951308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:38.951308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:38.951308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:38.952101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:38.952101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:38.952101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:38.952101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:38.952101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:38.952101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:38.952101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:38.952101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:38.952101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:38.952101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:38.952101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 04:14:38.952101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:38.952101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 204/403] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Aug 23 04:14:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:38.958801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877b858-b3cd-4c24-aefe-34f02b78be81 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:39.061538 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c362c8-034f-4826-80e9-a836e2863a0c None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:39.083693 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:39.101631 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877b858-b3cd-4c24-aefe-34f02b78be81 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:39.106614 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df453298-4976-4312-bb32-f3e145651beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:39.107268 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df453298-4976-4312-bb32-f3e145651beb None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:39.107551 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df453298-4976-4312-bb32-f3e145651beb None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:39.107825 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df453298-4976-4312-bb32-f3e145651beb None admin] PATH_INFO: `/v3/groups/7098ad8275714c318bdc7126575c8ceb` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:39.108990 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df453298-4976-4312-bb32-f3e145651beb None admin] RBAC: Authorizing `identity:get_group(group_id=7098ad8275714c318bdc7126575c8ceb)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:39.110446 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df453298-4976-4312-bb32-f3e145651beb None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:39.112857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 200/404] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Aug 23 04:14:38 2022] GET /identity/v3/groups/7098ad8275714c318bdc7126575c8ceb => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:39.114008 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df453298-4976-4312-bb32-f3e145651beb None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:39.140577 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a057a8-13c1-4c8c-861a-269cd729acec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:39.141470 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a057a8-13c1-4c8c-861a-269cd729acec None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:39.141958 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a057a8-13c1-4c8c-861a-269cd729acec None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:39.142410 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a057a8-13c1-4c8c-861a-269cd729acec None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:39.143244 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a057a8-13c1-4c8c-861a-269cd729acec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:39.143720 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a057a8-13c1-4c8c-861a-269cd729acec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:39.145527 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a057a8-13c1-4c8c-861a-269cd729acec None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:39.154848 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 205/405] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:14:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:39.156145 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a057a8-13c1-4c8c-861a-269cd729acec None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:39.185608 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:39.233775 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df453298-4976-4312-bb32-f3e145651beb None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:39.269197 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92626e-962d-4ffa-b1a1-14134f0b9f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:39.270228 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92626e-962d-4ffa-b1a1-14134f0b9f4e None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:39.270715 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92626e-962d-4ffa-b1a1-14134f0b9f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:39.271201 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92626e-962d-4ffa-b1a1-14134f0b9f4e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:39.279260 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a057a8-13c1-4c8c-861a-269cd729acec None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:39.283175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92626e-962d-4ffa-b1a1-14134f0b9f4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:39.288986 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-1e92626e-962d-4ffa-b1a1-14134f0b9f4e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:14:39.288986 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:39.288986 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:39.288986 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:39.288986 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:39.288986 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:39.288986 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:39.288986 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:39.288986 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:39.288986 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:39.288986 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:39.288986 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:39.288986 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:39.288986 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:39.288986 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:39.288986 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:39.289872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:39.289872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:39.289872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:39.289872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:39.289872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:39.289872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:39.289872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:39.289872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:39.289872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:39.289872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:39.289872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:39.289872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:39.289872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:39.289872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:39.289872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:39.289872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:39.290633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:39.290633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:39.290633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:39.290633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:39.290633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:39.290633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:39.290633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:39.290633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:39.290633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:39.290633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:39.290633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:14:39.290633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:39.293026 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 201/406] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:14:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:39.294066 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92626e-962d-4ffa-b1a1-14134f0b9f4e None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:39.323264 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b16868-2791-4c39-8c9d-5917656e3365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:39.324044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b16868-2791-4c39-8c9d-5917656e3365 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:39.324332 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b16868-2791-4c39-8c9d-5917656e3365 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:39.324742 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b16868-2791-4c39-8c9d-5917656e3365 None admin] PATH_INFO: `/v3/projects/95a15538272e426ba99c0b50e2f30f52` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:39.326190 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b16868-2791-4c39-8c9d-5917656e3365 None admin] RBAC: Authorizing `identity:get_project(project_id=95a15538272e426ba99c0b50e2f30f52)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:39.328217 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b16868-2791-4c39-8c9d-5917656e3365 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:39.330465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 206/407] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:39 2022] GET /identity/v3/projects/95a15538272e426ba99c0b50e2f30f52 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:39.331248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b16868-2791-4c39-8c9d-5917656e3365 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:39.349064 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:39.409566 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92626e-962d-4ffa-b1a1-14134f0b9f4e None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:39.428203 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:39.437073 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b16868-2791-4c39-8c9d-5917656e3365 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:39.448458 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b8ccd-bcbe-4b8f-82eb-fa34e8202513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:39.449377 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b8ccd-bcbe-4b8f-82eb-fa34e8202513 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:39.449839 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b8ccd-bcbe-4b8f-82eb-fa34e8202513 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:39.450272 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b8ccd-bcbe-4b8f-82eb-fa34e8202513 None admin] PATH_INFO: `/v3/groups/7098ad8275714c318bdc7126575c8ceb` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:39.451688 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7b8ccd-bcbe-4b8f-82eb-fa34e8202513 None admin] RBAC: Authorizing `identity:get_group(group_id=7098ad8275714c318bdc7126575c8ceb)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:39.453474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7b8ccd-bcbe-4b8f-82eb-fa34e8202513 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:39.455738 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 202/408] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Aug 23 04:14:39 2022] GET /identity/v3/groups/7098ad8275714c318bdc7126575c8ceb => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:39.456755 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b8ccd-bcbe-4b8f-82eb-fa34e8202513 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:39.479422 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089738e3-597d-4238-8af8-6c71035b613e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:39.480103 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089738e3-597d-4238-8af8-6c71035b613e None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:39.480386 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089738e3-597d-4238-8af8-6c71035b613e None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:39.480651 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089738e3-597d-4238-8af8-6c71035b613e None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:39.481319 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089738e3-597d-4238-8af8-6c71035b613e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:39.481619 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089738e3-597d-4238-8af8-6c71035b613e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:39.482990 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089738e3-597d-4238-8af8-6c71035b613e None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:39.492102 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 207/409] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:39.493009 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089738e3-597d-4238-8af8-6c71035b613e None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:39.567718 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b8ccd-bcbe-4b8f-82eb-fa34e8202513 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:39.582952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:39.598343 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089738e3-597d-4238-8af8-6c71035b613e None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:39.608779 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ecaa7-141d-43d7-af65-6e2c9a694ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:39.609540 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ecaa7-141d-43d7-af65-6e2c9a694ef6 None admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:39.609826 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ecaa7-141d-43d7-af65-6e2c9a694ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:39.610158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ecaa7-141d-43d7-af65-6e2c9a694ef6 None admin] PATH_INFO: `/v3/projects/95a15538272e426ba99c0b50e2f30f52/groups/7098ad8275714c318bdc7126575c8ceb/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:39.612210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ecaa7-141d-43d7-af65-6e2c9a694ef6 None admin] RBAC: Authorizing `identity:create_grant(project_id=95a15538272e426ba99c0b50e2f30f52, group_id=7098ad8275714c318bdc7126575c8ceb, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:39.613826 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ecaa7-141d-43d7-af65-6e2c9a694ef6 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:39.636026 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 203/410] 10.222.0.18 () {58 vars in 1442 bytes} [Tue Aug 23 04:14:39 2022] PUT /identity/v3/projects/95a15538272e426ba99c0b50e2f30f52/groups/7098ad8275714c318bdc7126575c8ceb/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:14:39.637018 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ecaa7-141d-43d7-af65-6e2c9a694ef6 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:39.646905 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f53a68-8113-4c32-9472-7e5053497c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:39.650035 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f53a68-8113-4c32-9472-7e5053497c91 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:39.650369 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f53a68-8113-4c32-9472-7e5053497c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:39.650634 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f53a68-8113-4c32-9472-7e5053497c91 None admin] PATH_INFO: `/v3/projects/95a15538272e426ba99c0b50e2f30f52` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:39.653115 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f53a68-8113-4c32-9472-7e5053497c91 None admin] RBAC: Authorizing `identity:get_project(project_id=95a15538272e426ba99c0b50e2f30f52)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:39.657850 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f53a68-8113-4c32-9472-7e5053497c91 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:39.663773 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 208/411] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:39 2022] GET /identity/v3/projects/95a15538272e426ba99c0b50e2f30f52 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:39.666673 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f53a68-8113-4c32-9472-7e5053497c91 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:39.729335 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:39.822136 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ecaa7-141d-43d7-af65-6e2c9a694ef6 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:39.858365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f53a68-8113-4c32-9472-7e5053497c91 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:39.872607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341d339f-8e77-475f-b72f-8eda93f6f7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:39.873780 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d339f-8e77-475f-b72f-8eda93f6f7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:39.874565 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d339f-8e77-475f-b72f-8eda93f6f7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:39.874842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d339f-8e77-475f-b72f-8eda93f6f7e0 None admin] PATH_INFO: `/v3/groups/8c7014f78086414a80a7008cf794a331` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:39.876854 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341d339f-8e77-475f-b72f-8eda93f6f7e0 None admin] RBAC: Authorizing `identity:get_group(group_id=8c7014f78086414a80a7008cf794a331)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:39.878780 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341d339f-8e77-475f-b72f-8eda93f6f7e0 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:39.881399 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 204/412] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Aug 23 04:14:39 2022] GET /identity/v3/groups/8c7014f78086414a80a7008cf794a331 => generated 229 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:39.886526 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341d339f-8e77-475f-b72f-8eda93f6f7e0 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:39.896507 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734f3c0-6c5c-4271-bb89-1e733c0c0129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:39.897171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734f3c0-6c5c-4271-bb89-1e733c0c0129 None admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:39.897508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734f3c0-6c5c-4271-bb89-1e733c0c0129 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:39.897779 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734f3c0-6c5c-4271-bb89-1e733c0c0129 None admin] PATH_INFO: `/v3/projects/95a15538272e426ba99c0b50e2f30f52/groups/7098ad8275714c318bdc7126575c8ceb/roles/4207a743edc04b0aba9197fe3554b2e3` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:39.899948 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734f3c0-6c5c-4271-bb89-1e733c0c0129 None admin] RBAC: Authorizing `identity:create_grant(project_id=95a15538272e426ba99c0b50e2f30f52, group_id=7098ad8275714c318bdc7126575c8ceb, role_id=4207a743edc04b0aba9197fe3554b2e3)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:39.901424 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734f3c0-6c5c-4271-bb89-1e733c0c0129 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:39.919283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 209/413] 10.222.0.18 () {58 vars in 1442 bytes} [Tue Aug 23 04:14:39 2022] PUT /identity/v3/projects/95a15538272e426ba99c0b50e2f30f52/groups/7098ad8275714c318bdc7126575c8ceb/roles/4207a743edc04b0aba9197fe3554b2e3 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:14:39.939091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:40.009946 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341d339f-8e77-475f-b72f-8eda93f6f7e0 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:40.047513 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35823357-1063-4241-99cb-ca728e6aa252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:40.048207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35823357-1063-4241-99cb-ca728e6aa252 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:40.048481 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35823357-1063-4241-99cb-ca728e6aa252 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:40.048752 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35823357-1063-4241-99cb-ca728e6aa252 None admin] PATH_INFO: `/v3/projects/ba69f2ba4f0d496e9c7607a800699841` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:40.049971 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35823357-1063-4241-99cb-ca728e6aa252 None admin] RBAC: Authorizing `identity:get_project(project_id=ba69f2ba4f0d496e9c7607a800699841)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:40.051539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35823357-1063-4241-99cb-ca728e6aa252 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:40.053877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 205/414] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:39 2022] GET /identity/v3/projects/ba69f2ba4f0d496e9c7607a800699841 => generated 344 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:40.059536 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734f3c0-6c5c-4271-bb89-1e733c0c0129 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:40.158944 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:40.181277 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734f3c0-6c5c-4271-bb89-1e733c0c0129 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:40.224142 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e44d1d9-95e3-40ec-853f-594774923647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:40.224856 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44d1d9-95e3-40ec-853f-594774923647 None admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:40.225135 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44d1d9-95e3-40ec-853f-594774923647 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:40.225429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44d1d9-95e3-40ec-853f-594774923647 None admin] PATH_INFO: `/v3/projects/ba69f2ba4f0d496e9c7607a800699841/groups/8c7014f78086414a80a7008cf794a331/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:40.228448 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e44d1d9-95e3-40ec-853f-594774923647 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba69f2ba4f0d496e9c7607a800699841, group_id=8c7014f78086414a80a7008cf794a331, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:40.229981 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e44d1d9-95e3-40ec-853f-594774923647 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:40.254678 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 210/415] 10.222.0.18 () {58 vars in 1442 bytes} [Tue Aug 23 04:14:40 2022] PUT /identity/v3/projects/ba69f2ba4f0d496e9c7607a800699841/groups/8c7014f78086414a80a7008cf794a331/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:14:40.599448 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4bfb75-7275-4500-8efe-bb8f6add9f51 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:40.600156 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4bfb75-7275-4500-8efe-bb8f6add9f51 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:40.600760 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4bfb75-7275-4500-8efe-bb8f6add9f51 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:40.602320 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 206/416] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:40.608909 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022ed1f-42bf-4cd9-ab1b-387fa8508e09 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:40.609445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022ed1f-42bf-4cd9-ab1b-387fa8508e09 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:40.609907 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022ed1f-42bf-4cd9-ab1b-387fa8508e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:40.632926 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737cddf-7945-497d-a158-49645686b51c None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:40.633226 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737cddf-7945-497d-a158-49645686b51c None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:40.633486 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737cddf-7945-497d-a158-49645686b51c None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:40.633837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-4022ed1f-42bf-4cd9-ab1b-387fa8508e09 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:40.635068 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 207/417] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:40.635484 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-4022ed1f-42bf-4cd9-ab1b-387fa8508e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:40.641020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c7b5c-cdd5-47a5-9e90-c8ac804accde None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:40.641313 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c7b5c-cdd5-47a5-9e90-c8ac804accde None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:40.641591 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c7b5c-cdd5-47a5-9e90-c8ac804accde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:40.663953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-a44c7b5c-cdd5-47a5-9e90-c8ac804accde None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:40.663953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a44c7b5c-cdd5-47a5-9e90-c8ac804accde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:40.750195 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 211/418] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:40.775235 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 208/419] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 04:14:41.151973 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4022ed1f-42bf-4cd9-ab1b-387fa8508e09 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:41.166908 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c7b5c-cdd5-47a5-9e90-c8ac804accde None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:41.244462 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:41.272138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:41.333414 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4022ed1f-42bf-4cd9-ab1b-387fa8508e09 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:41.361486 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c7b5c-cdd5-47a5-9e90-c8ac804accde None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:41.395634 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3afcb3-8239-4e5e-aa20-d0332410f030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:41.396300 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3afcb3-8239-4e5e-aa20-d0332410f030 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:41.396581 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3afcb3-8239-4e5e-aa20-d0332410f030 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:41.396853 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3afcb3-8239-4e5e-aa20-d0332410f030 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:41.413377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3afcb3-8239-4e5e-aa20-d0332410f030 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:41.418878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-fb3afcb3-8239-4e5e-aa20-d0332410f030 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:41.418878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:41.418878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:41.418878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:41.418878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:41.418878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:41.418878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:41.418878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:41.418878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:41.418878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:41.418878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:41.418878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:41.418878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:41.418878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:41.418878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:41.418878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:41.419727 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:41.419727 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:41.419727 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:41.419727 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:41.419727 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:41.419727 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:41.419727 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:41.419727 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:41.419727 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:41.419727 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:41.419727 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:41.419727 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:41.419727 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:41.419727 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:41.419727 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:41.419727 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:41.420538 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:41.420538 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:41.420538 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:41.420538 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:41.420538 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:41.420538 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:41.420538 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:41.420538 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:41.420538 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:41.420538 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:41.420538 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:41.420538 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:41.421117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 212/420] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:41.427055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3afcb3-8239-4e5e-aa20-d0332410f030 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:41.430763 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3c41b-a24b-46f1-9605-b7542d536e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:41.431340 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3c41b-a24b-46f1-9605-b7542d536e05 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:41.431617 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3c41b-a24b-46f1-9605-b7542d536e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:41.431891 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3c41b-a24b-46f1-9605-b7542d536e05 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:41.466047 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3c41b-a24b-46f1-9605-b7542d536e05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:41.471746 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-5ca3c41b-a24b-46f1-9605-b7542d536e05 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 04:14:41.471746 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:41.471746 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:41.471746 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:41.471746 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:41.471746 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:41.471746 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:41.471746 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:41.471746 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:41.471746 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:41.471746 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:41.471746 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:41.471746 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:41.471746 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:41.471746 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:41.471746 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:41.472634 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:41.472634 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:41.472634 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:41.472634 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:41.472634 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:41.472634 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:41.472634 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:41.472634 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:41.472634 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:41.472634 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:41.472634 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:41.472634 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:41.472634 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:41.472634 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:41.472634 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:41.472634 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:41.473392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:41.473392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:41.473392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:41.473392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:41.473392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:41.473392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:41.473392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:41.473392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:41.473392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:41.473392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:41.473392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 04:14:41.473392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:41.480546 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 209/421] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Aug 23 04:14:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:41.484435 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3c41b-a24b-46f1-9605-b7542d536e05 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:41.627228 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3afcb3-8239-4e5e-aa20-d0332410f030 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:41.664059 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:41.672378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:41.674518 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5134514-6715-4a45-876c-f20a42f254a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:41.675428 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5134514-6715-4a45-876c-f20a42f254a2 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:41.675428 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5134514-6715-4a45-876c-f20a42f254a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:41.675673 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5134514-6715-4a45-876c-f20a42f254a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:41.676408 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5134514-6715-4a45-876c-f20a42f254a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:41.676716 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5134514-6715-4a45-876c-f20a42f254a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:41.678166 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5134514-6715-4a45-876c-f20a42f254a2 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:41.681957 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3c41b-a24b-46f1-9605-b7542d536e05 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:41.706862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 213/422] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Aug 23 04:14:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:41.712707 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5134514-6715-4a45-876c-f20a42f254a2 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:41.782312 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9dbfa-deb0-4191-b699-23d10344ea13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:41.782958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9dbfa-deb0-4191-b699-23d10344ea13 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:41.783235 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9dbfa-deb0-4191-b699-23d10344ea13 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:41.783504 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9dbfa-deb0-4191-b699-23d10344ea13 None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:41.784208 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba9dbfa-deb0-4191-b699-23d10344ea13 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:41.784514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba9dbfa-deb0-4191-b699-23d10344ea13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:41.785900 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba9dbfa-deb0-4191-b699-23d10344ea13 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:41.812915 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 210/423] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:14:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:41.821323 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9dbfa-deb0-4191-b699-23d10344ea13 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:41.868541 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5134514-6715-4a45-876c-f20a42f254a2 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:41.907236 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb16c332-904d-405f-9005-70a877757f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:41.908180 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16c332-904d-405f-9005-70a877757f74 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:41.908581 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16c332-904d-405f-9005-70a877757f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:41.908970 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16c332-904d-405f-9005-70a877757f74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:41.996868 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:42.006301 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb16c332-904d-405f-9005-70a877757f74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:42.007873 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9dbfa-deb0-4191-b699-23d10344ea13 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:42.008124 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb16c332-904d-405f-9005-70a877757f74 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:42.045767 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38566f-2bdd-4fe6-8845-7df977e78251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:42.046385 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38566f-2bdd-4fe6-8845-7df977e78251 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:42.046659 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38566f-2bdd-4fe6-8845-7df977e78251 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:42.046925 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38566f-2bdd-4fe6-8845-7df977e78251 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:42.079990 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:42.122563 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed38566f-2bdd-4fe6-8845-7df977e78251 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:42.124062 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed38566f-2bdd-4fe6-8845-7df977e78251 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:42.132564 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 214/424] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:14:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:14:42.133979 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ca38d-0d12-4c24-a1c6-874d91cb16cc None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:42.134267 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ca38d-0d12-4c24-a1c6-874d91cb16cc None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:42.134923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ca38d-0d12-4c24-a1c6-874d91cb16cc None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:42.140529 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 215/425] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:42.140529 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247de1f-cd81-4dc5-aa2b-a88d15be7dc2 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:42.140529 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247de1f-cd81-4dc5-aa2b-a88d15be7dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:42.140529 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247de1f-cd81-4dc5-aa2b-a88d15be7dc2 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:42.144040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 216/426] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:42.145208 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247de1f-cd81-4dc5-aa2b-a88d15be7dc2 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:42.232445 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 211/427] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:14:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:14:42.234272 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee75c77-009e-4915-9974-63f2851cb920 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:42.234556 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee75c77-009e-4915-9974-63f2851cb920 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:42.234831 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee75c77-009e-4915-9974-63f2851cb920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:42.253485 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:42.253776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-9ee75c77-009e-4915-9974-63f2851cb920 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:42.255077 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-9ee75c77-009e-4915-9974-63f2851cb920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:42.262802 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247de1f-cd81-4dc5-aa2b-a88d15be7dc2 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:42.307578 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11eb45-a1fa-4eeb-9059-8138996f8018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:42.308276 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11eb45-a1fa-4eeb-9059-8138996f8018 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:42.308893 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11eb45-a1fa-4eeb-9059-8138996f8018 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:42.309167 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11eb45-a1fa-4eeb-9059-8138996f8018 None admin] PATH_INFO: `/v3/projects/e3148f82c47e4702b119cf867f5dcdcc` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:42.310303 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f11eb45-a1fa-4eeb-9059-8138996f8018 None admin] RBAC: Authorizing `identity:get_project(project_id=e3148f82c47e4702b119cf867f5dcdcc)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:42.312308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f11eb45-a1fa-4eeb-9059-8138996f8018 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:42.314408 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:42.316035 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 217/428] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:42 2022] GET /identity/v3/projects/e3148f82c47e4702b119cf867f5dcdcc => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:42.317290 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443d33c-02fe-46b1-b525-014400a85a4d None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:42.317574 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443d33c-02fe-46b1-b525-014400a85a4d None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:42.317904 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443d33c-02fe-46b1-b525-014400a85a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:42.334290 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 212/429] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:42.335604 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee75c77-009e-4915-9974-63f2851cb920 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:42.345651 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-3443d33c-02fe-46b1-b525-014400a85a4d None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:42.349206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-3443d33c-02fe-46b1-b525-014400a85a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:42.430656 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 218/430] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:42.432261 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443d33c-02fe-46b1-b525-014400a85a4d None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:42.444669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee75c77-009e-4915-9974-63f2851cb920 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:42.520215 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252144dc-c5df-4b6a-abbe-f718dc4aca27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:42.521064 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252144dc-c5df-4b6a-abbe-f718dc4aca27 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:42.521342 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252144dc-c5df-4b6a-abbe-f718dc4aca27 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:42.521610 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252144dc-c5df-4b6a-abbe-f718dc4aca27 None admin] PATH_INFO: `/v3/projects/e3148f82c47e4702b119cf867f5dcdcc` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:42.525281 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252144dc-c5df-4b6a-abbe-f718dc4aca27 None admin] RBAC: Authorizing `identity:get_project(project_id=e3148f82c47e4702b119cf867f5dcdcc)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:42.526845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252144dc-c5df-4b6a-abbe-f718dc4aca27 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:42.528598 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 213/431] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:42 2022] GET /identity/v3/projects/e3148f82c47e4702b119cf867f5dcdcc => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:42.534120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252144dc-c5df-4b6a-abbe-f718dc4aca27 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:42.536517 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:42.546905 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443d33c-02fe-46b1-b525-014400a85a4d None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:42.584674 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f056f-5831-41e8-8767-5ccc2275243e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:42.585960 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f056f-5831-41e8-8767-5ccc2275243e None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:42.586597 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f056f-5831-41e8-8767-5ccc2275243e None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:42.587072 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f056f-5831-41e8-8767-5ccc2275243e None admin] PATH_INFO: `/v3/groups/7098ad8275714c318bdc7126575c8ceb` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:42.589471 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7f056f-5831-41e8-8767-5ccc2275243e None admin] RBAC: Authorizing `identity:get_group(group_id=7098ad8275714c318bdc7126575c8ceb)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:42.591359 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7f056f-5831-41e8-8767-5ccc2275243e None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:42.593437 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 219/432] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Aug 23 04:14:42 2022] GET /identity/v3/groups/7098ad8275714c318bdc7126575c8ceb => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:42.599550 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f056f-5831-41e8-8767-5ccc2275243e None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:42.651539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:42.701376 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252144dc-c5df-4b6a-abbe-f718dc4aca27 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:42.717267 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f056f-5831-41e8-8767-5ccc2275243e None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:42.736995 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f0f71-e252-4fd5-bfea-b0c19dc909fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:42.737630 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f0f71-e252-4fd5-bfea-b0c19dc909fe None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:42.737903 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f0f71-e252-4fd5-bfea-b0c19dc909fe None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:42.738170 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f0f71-e252-4fd5-bfea-b0c19dc909fe None admin] PATH_INFO: `/v3/groups/7098ad8275714c318bdc7126575c8ceb` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:42.739253 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f0f71-e252-4fd5-bfea-b0c19dc909fe None admin] RBAC: Authorizing `identity:get_group(group_id=7098ad8275714c318bdc7126575c8ceb)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:42.740716 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f0f71-e252-4fd5-bfea-b0c19dc909fe None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:42.742470 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 214/433] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Aug 23 04:14:42 2022] GET /identity/v3/groups/7098ad8275714c318bdc7126575c8ceb => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:14:42.743306 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f0f71-e252-4fd5-bfea-b0c19dc909fe None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:42.753544 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9d7a3-ca3a-42d7-b3eb-2f4ee3a957f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:42.754144 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9d7a3-ca3a-42d7-b3eb-2f4ee3a957f7 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:42.754402 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9d7a3-ca3a-42d7-b3eb-2f4ee3a957f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:42.754683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9d7a3-ca3a-42d7-b3eb-2f4ee3a957f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:42.755404 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9d7a3-ca3a-42d7-b3eb-2f4ee3a957f7 None admin] RBAC: Adding query filter params (group.id=7098ad8275714c318bdc7126575c8ceb, role.id=aa03504b4fb642d288d07f4335e291e1, scope.project.id=e3148f82c47e4702b119cf867f5dcdcc) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:42.755704 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9d7a3-ca3a-42d7-b3eb-2f4ee3a957f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:42.757477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9d7a3-ca3a-42d7-b3eb-2f4ee3a957f7 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:42.763092 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 220/434] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Aug 23 04:14:42 2022] GET /identity/v3/role_assignments?group.id=7098ad8275714c318bdc7126575c8ceb&scope.project.id=e3148f82c47e4702b119cf867f5dcdcc&role.id=aa03504b4fb642d288d07f4335e291e1 => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:14:42.764357 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986989f4-8930-4d20-a728-c02d6716b4f9 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:42.764641 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986989f4-8930-4d20-a728-c02d6716b4f9 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:42.764908 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986989f4-8930-4d20-a728-c02d6716b4f9 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:42.766077 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 221/435] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:42.766777 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986989f4-8930-4d20-a728-c02d6716b4f9 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:42.794391 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:42.863613 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f0f71-e252-4fd5-bfea-b0c19dc909fe None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:42.884870 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986989f4-8930-4d20-a728-c02d6716b4f9 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:42.893183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:42.904745 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e810cff4-8a5f-4d54-8878-a4ce85948c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:42.905398 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e810cff4-8a5f-4d54-8878-a4ce85948c58 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:42.905674 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e810cff4-8a5f-4d54-8878-a4ce85948c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:42.906014 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e810cff4-8a5f-4d54-8878-a4ce85948c58 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:42.920785 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e810cff4-8a5f-4d54-8878-a4ce85948c58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:42.925522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-e810cff4-8a5f-4d54-8878-a4ce85948c58 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:42.925522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:42.925522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:42.925522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:42.925522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:42.925522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:42.925522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:42.925522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:42.925522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:42.925522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:42.925522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:42.925522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:42.925522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:42.925522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:42.925522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:42.925522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:42.926396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:42.926396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:42.926396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:42.926396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:42.926396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:42.926396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:42.926396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:42.926396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:42.926396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:42.926396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:42.926396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:42.926396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:42.926396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:42.926396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:42.926396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:42.926396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:42.927171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:42.927171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:42.927171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:42.927171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:42.927171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:42.927171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:42.927171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:42.927171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:42.927171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:42.927171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:42.927171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 04:14:42.927171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:42.927171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 215/436] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:42.927801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e584b-1a0f-4a76-b424-7f9489382e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:42.927897 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e810cff4-8a5f-4d54-8878-a4ce85948c58 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:42.932638 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e584b-1a0f-4a76-b424-7f9489382e06 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:42.932638 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e584b-1a0f-4a76-b424-7f9489382e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:42.932638 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e584b-1a0f-4a76-b424-7f9489382e06 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:42.945139 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8e584b-1a0f-4a76-b424-7f9489382e06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:42.950573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-da8e584b-1a0f-4a76-b424-7f9489382e06 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 04:14:42.950573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:42.950573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:42.950573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:42.950573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:42.950573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:42.950573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:42.950573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:42.950573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:42.950573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:42.950573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:42.950573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:42.950573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:42.950573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:42.950573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:42.950573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:42.951421 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:42.951421 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:42.951421 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:42.951421 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:42.951421 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:42.951421 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:42.951421 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:42.951421 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:42.951421 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:42.951421 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:42.951421 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:42.951421 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:42.951421 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:42.951421 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:42.951421 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:42.951421 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:42.952227 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:42.952227 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:42.952227 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:42.952227 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:42.952227 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:42.952227 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:42.952227 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:42.952227 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:42.952227 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:42.952227 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:42.952227 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 04:14:42.952227 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:42.952227 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 222/437] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Aug 23 04:14:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:42.953247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f681383-86a4-4c40-a498-098b4310a46e None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:42.953732 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f681383-86a4-4c40-a498-098b4310a46e None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:42.953837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f681383-86a4-4c40-a498-098b4310a46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:42.978505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-0f681383-86a4-4c40-a498-098b4310a46e None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:42.979704 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-0f681383-86a4-4c40-a498-098b4310a46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:43.057597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e810cff4-8a5f-4d54-8878-a4ce85948c58 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:43.057769 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 223/438] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:43.058558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f681383-86a4-4c40-a498-098b4310a46e None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:43.078734 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:43.101130 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db5104-7cf3-4fb3-80b5-09fc3d2d0654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:43.101742 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db5104-7cf3-4fb3-80b5-09fc3d2d0654 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:43.102173 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db5104-7cf3-4fb3-80b5-09fc3d2d0654 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:43.102610 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db5104-7cf3-4fb3-80b5-09fc3d2d0654 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:43.103482 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78db5104-7cf3-4fb3-80b5-09fc3d2d0654 None admin] RBAC: Adding query filter params (group.id=7098ad8275714c318bdc7126575c8ceb, role.id=4207a743edc04b0aba9197fe3554b2e3, scope.project.id=e3148f82c47e4702b119cf867f5dcdcc) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:43.103952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78db5104-7cf3-4fb3-80b5-09fc3d2d0654 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:43.105528 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78db5104-7cf3-4fb3-80b5-09fc3d2d0654 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:43.115399 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 216/439] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Aug 23 04:14:42 2022] GET /identity/v3/role_assignments?group.id=7098ad8275714c318bdc7126575c8ceb&scope.project.id=e3148f82c47e4702b119cf867f5dcdcc&role.id=4207a743edc04b0aba9197fe3554b2e3 => generated 611 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:43.116590 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db5104-7cf3-4fb3-80b5-09fc3d2d0654 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:43.186296 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:43.215036 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f681383-86a4-4c40-a498-098b4310a46e None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:43.259908 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b665da-0413-47d0-b356-9c9003310331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:43.260574 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b665da-0413-47d0-b356-9c9003310331 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:43.260855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b665da-0413-47d0-b356-9c9003310331 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:43.261125 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b665da-0413-47d0-b356-9c9003310331 None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:43.261810 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b665da-0413-47d0-b356-9c9003310331 None admin] RBAC: Adding query filter params (name=member) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:43.262111 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b665da-0413-47d0-b356-9c9003310331 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:43.263827 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b665da-0413-47d0-b356-9c9003310331 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:43.273404 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db5104-7cf3-4fb3-80b5-09fc3d2d0654 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:43.274867 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 224/440] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Aug 23 04:14:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:43.283370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b665da-0413-47d0-b356-9c9003310331 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:43.314315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb4f0f-2e32-4951-9e9e-c5cb28fa90e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:43.315021 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb4f0f-2e32-4951-9e9e-c5cb28fa90e8 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:43.315299 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb4f0f-2e32-4951-9e9e-c5cb28fa90e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:43.315569 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb4f0f-2e32-4951-9e9e-c5cb28fa90e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:43.316296 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcb4f0f-2e32-4951-9e9e-c5cb28fa90e8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:43.316564 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcb4f0f-2e32-4951-9e9e-c5cb28fa90e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:43.317928 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcb4f0f-2e32-4951-9e9e-c5cb28fa90e8 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:43.328635 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 217/441] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:14:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:43.334924 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb4f0f-2e32-4951-9e9e-c5cb28fa90e8 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:43.384005 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:43.410447 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b665da-0413-47d0-b356-9c9003310331 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:43.446913 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80de8c5c-10e2-48a6-9933-db809263c8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:43.447686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de8c5c-10e2-48a6-9933-db809263c8a9 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:43.448063 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de8c5c-10e2-48a6-9933-db809263c8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:43.448341 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de8c5c-10e2-48a6-9933-db809263c8a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:43.461627 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb4f0f-2e32-4951-9e9e-c5cb28fa90e8 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:43.479766 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:43.502637 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c406a48-1fdb-4a69-abd2-b5b6fb8d4607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:43.503957 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c406a48-1fdb-4a69-abd2-b5b6fb8d4607 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:43.503957 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c406a48-1fdb-4a69-abd2-b5b6fb8d4607 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:43.503957 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c406a48-1fdb-4a69-abd2-b5b6fb8d4607 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:43.543656 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80de8c5c-10e2-48a6-9933-db809263c8a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:43.545202 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80de8c5c-10e2-48a6-9933-db809263c8a9 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:43.576240 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c406a48-1fdb-4a69-abd2-b5b6fb8d4607 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:43.577608 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c406a48-1fdb-4a69-abd2-b5b6fb8d4607 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:43.620822 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:43.644817 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 225/442] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:14:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 04:14:43.645637 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80de8c5c-10e2-48a6-9933-db809263c8a9 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:43.675085 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 218/443] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:14:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:14:43.675927 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c406a48-1fdb-4a69-abd2-b5b6fb8d4607 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:43.689437 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:43.748272 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80de8c5c-10e2-48a6-9933-db809263c8a9 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:43.773070 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c406a48-1fdb-4a69-abd2-b5b6fb8d4607 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:43.781121 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8effde1a-41c1-4aae-a849-8e8323aa0c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:43.781742 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effde1a-41c1-4aae-a849-8e8323aa0c81 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:43.782021 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effde1a-41c1-4aae-a849-8e8323aa0c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:43.782284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effde1a-41c1-4aae-a849-8e8323aa0c81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:43.792131 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8effde1a-41c1-4aae-a849-8e8323aa0c81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:43.796663 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-8effde1a-41c1-4aae-a849-8e8323aa0c81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:14:43.796663 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:43.796663 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:43.796663 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:43.796663 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:43.796663 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:43.796663 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:43.796663 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:43.796663 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:43.796663 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:43.796663 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:43.796663 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:43.796663 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:43.796663 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:43.796663 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:43.796663 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:43.797477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:43.797477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:43.797477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:43.797477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:43.797477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:43.797477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:43.797477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:43.797477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:43.797477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:43.797477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:43.797477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:43.797477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:43.797477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:43.797477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:43.797477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:43.797477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:43.798248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:43.798248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:43.798248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:43.798248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:43.798248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:43.798248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:43.798248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:43.798248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:43.798248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:43.798248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:43.798248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:14:43.798248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:43.798248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 226/444] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:14:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:43.799139 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8effde1a-41c1-4aae-a849-8e8323aa0c81 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:43.808773 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157a92d-2d57-4a40-a9e2-836d6e726944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:43.809410 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157a92d-2d57-4a40-a9e2-836d6e726944 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:43.809682 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157a92d-2d57-4a40-a9e2-836d6e726944 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:43.809946 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157a92d-2d57-4a40-a9e2-836d6e726944 None admin] PATH_INFO: `/v3/projects/95a15538272e426ba99c0b50e2f30f52` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:43.811117 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8157a92d-2d57-4a40-a9e2-836d6e726944 None admin] RBAC: Authorizing `identity:get_project(project_id=95a15538272e426ba99c0b50e2f30f52)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:43.812594 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8157a92d-2d57-4a40-a9e2-836d6e726944 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:43.814209 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 219/445] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:43 2022] GET /identity/v3/projects/95a15538272e426ba99c0b50e2f30f52 => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:43.814955 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157a92d-2d57-4a40-a9e2-836d6e726944 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:43.835194 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:43.895437 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:43.902158 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8effde1a-41c1-4aae-a849-8e8323aa0c81 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:43.915260 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157a92d-2d57-4a40-a9e2-836d6e726944 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:43.936285 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9988baf9-b4e4-470b-97a5-c8e63c451729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:43.936896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988baf9-b4e4-470b-97a5-c8e63c451729 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:43.937191 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988baf9-b4e4-470b-97a5-c8e63c451729 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:43.937452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988baf9-b4e4-470b-97a5-c8e63c451729 None admin] PATH_INFO: `/v3/projects/95a15538272e426ba99c0b50e2f30f52` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:43.938548 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9988baf9-b4e4-470b-97a5-c8e63c451729 None admin] RBAC: Authorizing `identity:get_project(project_id=95a15538272e426ba99c0b50e2f30f52)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:43.940024 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9988baf9-b4e4-470b-97a5-c8e63c451729 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:43.941782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 227/446] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:43 2022] GET /identity/v3/projects/95a15538272e426ba99c0b50e2f30f52 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:43.942358 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9988baf9-b4e4-470b-97a5-c8e63c451729 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:43.953978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52583c6d-2b1a-44b1-8ea9-cb98d11c948b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:43.954592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52583c6d-2b1a-44b1-8ea9-cb98d11c948b None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:43.954865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52583c6d-2b1a-44b1-8ea9-cb98d11c948b None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:43.955133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52583c6d-2b1a-44b1-8ea9-cb98d11c948b None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:43.955797 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52583c6d-2b1a-44b1-8ea9-cb98d11c948b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:43.956123 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52583c6d-2b1a-44b1-8ea9-cb98d11c948b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:43.957489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52583c6d-2b1a-44b1-8ea9-cb98d11c948b None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:43.965978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 220/447] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:43.966818 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52583c6d-2b1a-44b1-8ea9-cb98d11c948b None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:44.038819 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9988baf9-b4e4-470b-97a5-c8e63c451729 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:44.041113 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:44.063363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52583c6d-2b1a-44b1-8ea9-cb98d11c948b None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:44.074378 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e2c837-f9a4-48f3-9ac3-b4f6580e0587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:44.075128 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2c837-f9a4-48f3-9ac3-b4f6580e0587 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:44.075421 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2c837-f9a4-48f3-9ac3-b4f6580e0587 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:44.075687 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2c837-f9a4-48f3-9ac3-b4f6580e0587 None admin] PATH_INFO: `/v3/groups/7098ad8275714c318bdc7126575c8ceb` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:44.076845 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e2c837-f9a4-48f3-9ac3-b4f6580e0587 None admin] RBAC: Authorizing `identity:get_group(group_id=7098ad8275714c318bdc7126575c8ceb)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:44.078295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e2c837-f9a4-48f3-9ac3-b4f6580e0587 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:44.079882 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 228/448] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Aug 23 04:14:43 2022] GET /identity/v3/groups/7098ad8275714c318bdc7126575c8ceb => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:44.080672 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e2c837-f9a4-48f3-9ac3-b4f6580e0587 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:44.096575 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff708f73-2907-4a8f-943c-906dff2f9bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:44.097136 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff708f73-2907-4a8f-943c-906dff2f9bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:44.097419 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff708f73-2907-4a8f-943c-906dff2f9bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:44.097690 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff708f73-2907-4a8f-943c-906dff2f9bb3 None admin] PATH_INFO: `/v3/groups/7098ad8275714c318bdc7126575c8ceb` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:44.098788 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff708f73-2907-4a8f-943c-906dff2f9bb3 None admin] RBAC: Authorizing `identity:get_group(group_id=7098ad8275714c318bdc7126575c8ceb)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:44.100256 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff708f73-2907-4a8f-943c-906dff2f9bb3 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:44.102022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 221/449] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Aug 23 04:14:43 2022] GET /identity/v3/groups/7098ad8275714c318bdc7126575c8ceb => generated 226 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:44.102769 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff708f73-2907-4a8f-943c-906dff2f9bb3 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:44.119289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:44.182642 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e2c837-f9a4-48f3-9ac3-b4f6580e0587 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:44.206824 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff708f73-2907-4a8f-943c-906dff2f9bb3 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:44.216308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048a170-fbd9-44c5-ada8-51161ad46fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:44.216909 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048a170-fbd9-44c5-ada8-51161ad46fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:44.217184 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048a170-fbd9-44c5-ada8-51161ad46fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:44.217457 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048a170-fbd9-44c5-ada8-51161ad46fe0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:44.242362 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12120e-4809-4938-8668-d0f10904c8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:44.242968 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12120e-4809-4938-8668-d0f10904c8d6 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:44.243245 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12120e-4809-4938-8668-d0f10904c8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:44.243531 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12120e-4809-4938-8668-d0f10904c8d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:44.244295 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12120e-4809-4938-8668-d0f10904c8d6 None admin] RBAC: Adding query filter params (group.id=7098ad8275714c318bdc7126575c8ceb, role.id=aa03504b4fb642d288d07f4335e291e1, scope.project.id=95a15538272e426ba99c0b50e2f30f52) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:44.244600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12120e-4809-4938-8668-d0f10904c8d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:44.246027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12120e-4809-4938-8668-d0f10904c8d6 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:44.251607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 222/450] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Aug 23 04:14:44 2022] GET /identity/v3/role_assignments?group.id=7098ad8275714c318bdc7126575c8ceb&scope.project.id=95a15538272e426ba99c0b50e2f30f52&role.id=aa03504b4fb642d288d07f4335e291e1 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:44.252396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12120e-4809-4938-8668-d0f10904c8d6 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:44.257751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:44.295348 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b048a170-fbd9-44c5-ada8-51161ad46fe0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:44.296880 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b048a170-fbd9-44c5-ada8-51161ad46fe0 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:44.358400 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:44.388312 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12120e-4809-4938-8668-d0f10904c8d6 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:44.416955 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 229/451] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:14:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:14:44.423652 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048a170-fbd9-44c5-ada8-51161ad46fe0 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:44.434876 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f4c4aa-0481-4379-a81b-d122033af2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:44.435497 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4c4aa-0481-4379-a81b-d122033af2a2 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:44.435763 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4c4aa-0481-4379-a81b-d122033af2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:44.436100 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4c4aa-0481-4379-a81b-d122033af2a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:44.436821 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f4c4aa-0481-4379-a81b-d122033af2a2 None admin] RBAC: Adding query filter params (group.id=7098ad8275714c318bdc7126575c8ceb, role.id=4207a743edc04b0aba9197fe3554b2e3, scope.project.id=95a15538272e426ba99c0b50e2f30f52) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:44.437122 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f4c4aa-0481-4379-a81b-d122033af2a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:44.438561 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f4c4aa-0481-4379-a81b-d122033af2a2 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:44.445073 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 223/452] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Aug 23 04:14:44 2022] GET /identity/v3/role_assignments?group.id=7098ad8275714c318bdc7126575c8ceb&scope.project.id=95a15538272e426ba99c0b50e2f30f52&role.id=4207a743edc04b0aba9197fe3554b2e3 => generated 611 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:44.513905 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:44.547187 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048a170-fbd9-44c5-ada8-51161ad46fe0 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:44.587559 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55106b3-4221-4863-b7dc-1fbe30c850c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:44.588252 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55106b3-4221-4863-b7dc-1fbe30c850c9 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:44.588877 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55106b3-4221-4863-b7dc-1fbe30c850c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:44.589145 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55106b3-4221-4863-b7dc-1fbe30c850c9 None admin] PATH_INFO: `/v3/projects/ba69f2ba4f0d496e9c7607a800699841` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:44.590293 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55106b3-4221-4863-b7dc-1fbe30c850c9 None admin] RBAC: Authorizing `identity:get_project(project_id=ba69f2ba4f0d496e9c7607a800699841)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:44.591767 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55106b3-4221-4863-b7dc-1fbe30c850c9 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:44.593445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 230/453] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:14:44 2022] GET /identity/v3/projects/ba69f2ba4f0d496e9c7607a800699841 => generated 344 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:14:44.599238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f4c4aa-0481-4379-a81b-d122033af2a2 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:44.705686 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f4c4aa-0481-4379-a81b-d122033af2a2 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:44.739939 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047f46db-1fcf-4f8c-994f-d4f2fa873b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:44.740593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f46db-1fcf-4f8c-994f-d4f2fa873b9c None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:44.740868 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f46db-1fcf-4f8c-994f-d4f2fa873b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:44.741204 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f46db-1fcf-4f8c-994f-d4f2fa873b9c None admin] PATH_INFO: `/v3/groups/8c7014f78086414a80a7008cf794a331` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:44.742301 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047f46db-1fcf-4f8c-994f-d4f2fa873b9c None admin] RBAC: Authorizing `identity:get_group(group_id=8c7014f78086414a80a7008cf794a331)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:44.743737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047f46db-1fcf-4f8c-994f-d4f2fa873b9c None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:44.745344 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 224/454] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Aug 23 04:14:44 2022] GET /identity/v3/groups/8c7014f78086414a80a7008cf794a331 => generated 229 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:44.751273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55106b3-4221-4863-b7dc-1fbe30c850c9 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:44.850626 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55106b3-4221-4863-b7dc-1fbe30c850c9 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:44.883518 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a5af2-7319-4f08-a83a-ee3cdd0976ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:44.884141 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a5af2-7319-4f08-a83a-ee3cdd0976ee None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:44.884417 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a5af2-7319-4f08-a83a-ee3cdd0976ee None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:44.884686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a5af2-7319-4f08-a83a-ee3cdd0976ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:44.885409 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92a5af2-7319-4f08-a83a-ee3cdd0976ee None admin] RBAC: Adding query filter params (group.id=8c7014f78086414a80a7008cf794a331, role.id=52b31061b72c45ccb76dd26bf31f8d3f, scope.project.id=ba69f2ba4f0d496e9c7607a800699841) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:44.885714 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92a5af2-7319-4f08-a83a-ee3cdd0976ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:44.887149 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92a5af2-7319-4f08-a83a-ee3cdd0976ee None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:44.893015 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 231/455] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Aug 23 04:14:44 2022] GET /identity/v3/role_assignments?group.id=8c7014f78086414a80a7008cf794a331&scope.project.id=ba69f2ba4f0d496e9c7607a800699841&role.id=52b31061b72c45ccb76dd26bf31f8d3f => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:47.199718 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5358f-94b7-470b-bb7d-f9efd5493403 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:47.200315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5358f-94b7-470b-bb7d-f9efd5493403 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:47.200315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5358f-94b7-470b-bb7d-f9efd5493403 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:47.201954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 225/456] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:47.207215 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36419b-3e78-4807-b417-3ec32e4d2c13 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:47.207668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36419b-3e78-4807-b417-3ec32e4d2c13 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:47.207848 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36419b-3e78-4807-b417-3ec32e4d2c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:47.219683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:47.232983 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-8b36419b-3e78-4807-b417-3ec32e4d2c13 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:47.234181 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-8b36419b-3e78-4807-b417-3ec32e4d2c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:47.322389 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 232/457] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:47.512826 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf394e1-2406-4981-b369-0b01b48f9723 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:47.513154 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf394e1-2406-4981-b369-0b01b48f9723 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:47.513377 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf394e1-2406-4981-b369-0b01b48f9723 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:47.514998 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 226/458] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:47.519956 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc37023-8667-4539-8134-3787e1140df8 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:47.520267 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc37023-8667-4539-8134-3787e1140df8 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:47.520548 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc37023-8667-4539-8134-3787e1140df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:47.548976 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-5fc37023-8667-4539-8134-3787e1140df8 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:47.550193 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-5fc37023-8667-4539-8134-3787e1140df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:47.641557 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 233/459] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:47.650505 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e50dc3-024f-475f-9ee7-0fff2ef1582e None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:47.651040 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e50dc3-024f-475f-9ee7-0fff2ef1582e None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:47.651498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e50dc3-024f-475f-9ee7-0fff2ef1582e None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:47.653351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 227/460] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:47.658281 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b3fc3-7ef1-4272-b4ec-913a4af40b7a None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:47.658572 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b3fc3-7ef1-4272-b4ec-913a4af40b7a None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:47.658845 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b3fc3-7ef1-4272-b4ec-913a4af40b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:47.663626 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e50dc3-024f-475f-9ee7-0fff2ef1582e None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:47.677050 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:47.680576 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-ba8b3fc3-7ef1-4272-b4ec-913a4af40b7a None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:47.681934 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-ba8b3fc3-7ef1-4272-b4ec-913a4af40b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:47.776049 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:47.791443 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 234/461] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:47.825693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803d3b5-f8bf-41c0-b6a7-a5577060f5f1 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:47.825978 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803d3b5-f8bf-41c0-b6a7-a5577060f5f1 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:47.826256 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803d3b5-f8bf-41c0-b6a7-a5577060f5f1 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:47.827603 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 235/462] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:14:47.829845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e50dc3-024f-475f-9ee7-0fff2ef1582e None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:47.836663 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59013a16-7b14-4dc3-9895-14337006db9f None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:47.836947 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59013a16-7b14-4dc3-9895-14337006db9f None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:47.837221 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59013a16-7b14-4dc3-9895-14337006db9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:47.872848 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-59013a16-7b14-4dc3-9895-14337006db9f None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:47.874055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-59013a16-7b14-4dc3-9895-14337006db9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:47.895503 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c2084-2589-4dbc-8e3e-f191a6e1058a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:47.896165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c2084-2589-4dbc-8e3e-f191a6e1058a None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:47.896622 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c2084-2589-4dbc-8e3e-f191a6e1058a None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:47.897055 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c2084-2589-4dbc-8e3e-f191a6e1058a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:47.908264 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c2084-2589-4dbc-8e3e-f191a6e1058a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:47.909918 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c2084-2589-4dbc-8e3e-f191a6e1058a None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:47.925411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-577c2084-2589-4dbc-8e3e-f191a6e1058a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:47.925411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:47.925411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:47.925411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:47.925411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:47.925411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:47.925411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:47.925411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:47.925411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:47.925411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:47.925411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:47.925411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:47.925411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:14:47.925411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:14:47.925411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:14:47.925411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:14:47.926504 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:47.926504 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:47.926504 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:47.926504 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:47.926504 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:47.926504 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:47.926504 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:47.926504 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:47.926504 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:47.926504 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:47.926504 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:47.926504 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:47.926504 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:47.926504 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:47.926504 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:47.926504 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:47.926504 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:14:47.927531 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:14:47.927531 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:47.927531 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:47.930135 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 228/463] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:14:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:47.939958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c2084-2589-4dbc-8e3e-f191a6e1058a None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:47.989665 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 236/464] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 04:14:47.990693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59013a16-7b14-4dc3-9895-14337006db9f None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:48.061709 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:48.103286 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59013a16-7b14-4dc3-9895-14337006db9f None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:48.129556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:48.138922 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c2084-2589-4dbc-8e3e-f191a6e1058a None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:48.141964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966f2a7b-4796-4236-93b6-a94126f7f5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:48.142607 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f2a7b-4796-4236-93b6-a94126f7f5ff None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:48.142887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f2a7b-4796-4236-93b6-a94126f7f5ff None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:48.143160 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f2a7b-4796-4236-93b6-a94126f7f5ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:48.154028 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966f2a7b-4796-4236-93b6-a94126f7f5ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:48.155629 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966f2a7b-4796-4236-93b6-a94126f7f5ff None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:48.172329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-966f2a7b-4796-4236-93b6-a94126f7f5ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:48.172329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:48.172329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:48.172329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:48.172329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:48.172329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:48.172329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:48.172329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:48.172329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:48.172329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:48.172329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:48.172329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:48.172329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:14:48.172329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:14:48.172329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:14:48.172329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:14:48.173317 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:48.173317 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:48.173317 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:48.173317 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:48.173317 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:48.173317 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:48.173317 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:48.173317 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:48.173317 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:48.173317 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:48.173317 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:48.173317 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:48.173317 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:48.173317 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:48.173317 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:48.173317 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:48.173317 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:14:48.174746 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:14:48.174746 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:48.174746 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:48.174746 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 237/465] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:14:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:48.174746 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966f2a7b-4796-4236-93b6-a94126f7f5ff None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:48.177945 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a3e93-cbca-424f-bee1-aed2a4ef4272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:48.178982 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a3e93-cbca-424f-bee1-aed2a4ef4272 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:48.179473 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a3e93-cbca-424f-bee1-aed2a4ef4272 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:48.180083 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a3e93-cbca-424f-bee1-aed2a4ef4272 None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:48.180939 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a3e93-cbca-424f-bee1-aed2a4ef4272 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:48.181426 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a3e93-cbca-424f-bee1-aed2a4ef4272 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:48.183093 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a3e93-cbca-424f-bee1-aed2a4ef4272 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:48.196731 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 229/466] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:48.197859 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a3e93-cbca-424f-bee1-aed2a4ef4272 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:48.289525 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966f2a7b-4796-4236-93b6-a94126f7f5ff None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:48.318799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a3e93-cbca-424f-bee1-aed2a4ef4272 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:48.325333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf736c-c924-4411-90af-29b71c66b3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:48.326229 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf736c-c924-4411-90af-29b71c66b3c7 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:48.326708 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf736c-c924-4411-90af-29b71c66b3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:48.327043 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf736c-c924-4411-90af-29b71c66b3c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:48.340173 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:48.341055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf736c-c924-4411-90af-29b71c66b3c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:48.342315 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf736c-c924-4411-90af-29b71c66b3c7 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:48.357442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-0adf736c-c924-4411-90af-29b71c66b3c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:48.357442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:48.357442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:48.357442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:48.357442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:48.357442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:48.357442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:48.357442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:48.357442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:48.357442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:48.357442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:48.357442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:48.357442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:14:48.357442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:14:48.357442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:14:48.357442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:14:48.358439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:48.358439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:48.358439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:48.358439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:48.358439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:48.358439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:48.358439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:48.358439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:48.358439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:48.358439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:48.358439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:48.358439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:48.358439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:48.358439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:48.358439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:48.358439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:48.358439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:14:48.359248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:14:48.359248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:48.359248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:48.359248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 238/467] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:14:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:48.359457 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf736c-c924-4411-90af-29b71c66b3c7 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:48.362163 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eda0b7-cf6b-4e58-9b19-ccedaedaa73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:48.362799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eda0b7-cf6b-4e58-9b19-ccedaedaa73d None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:48.363058 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eda0b7-cf6b-4e58-9b19-ccedaedaa73d None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:48.363326 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eda0b7-cf6b-4e58-9b19-ccedaedaa73d None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:48.364263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eda0b7-cf6b-4e58-9b19-ccedaedaa73d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:48.364566 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eda0b7-cf6b-4e58-9b19-ccedaedaa73d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:48.365992 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eda0b7-cf6b-4e58-9b19-ccedaedaa73d None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:48.383090 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 230/468] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:48.383090 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eda0b7-cf6b-4e58-9b19-ccedaedaa73d None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:48.395934 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:48.473067 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf736c-c924-4411-90af-29b71c66b3c7 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:48.496132 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eda0b7-cf6b-4e58-9b19-ccedaedaa73d None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:48.508054 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600fd75c-781a-4c3e-b0fc-79c284c462f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:48.508676 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600fd75c-781a-4c3e-b0fc-79c284c462f6 None admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:48.508965 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600fd75c-781a-4c3e-b0fc-79c284c462f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:48.509229 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600fd75c-781a-4c3e-b0fc-79c284c462f6 None admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:48.509867 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600fd75c-781a-4c3e-b0fc-79c284c462f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:48.511370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600fd75c-781a-4c3e-b0fc-79c284c462f6 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:48.532772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6320ae61-63c1-418f-8d27-9a8e9f1ac3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:48.533406 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320ae61-63c1-418f-8d27-9a8e9f1ac3fc None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:48.533679 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320ae61-63c1-418f-8d27-9a8e9f1ac3fc None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:48.544202 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320ae61-63c1-418f-8d27-9a8e9f1ac3fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:48.544411 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 239/469] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Aug 23 04:14:48 2022] POST /identity/v3/users => generated 246 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:14:48.544411 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600fd75c-781a-4c3e-b0fc-79c284c462f6 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:48.557584 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6320ae61-63c1-418f-8d27-9a8e9f1ac3fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:48.559150 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6320ae61-63c1-418f-8d27-9a8e9f1ac3fc None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:48.575817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-6320ae61-63c1-418f-8d27-9a8e9f1ac3fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:48.575817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:48.575817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:48.575817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:48.575817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:48.575817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:48.575817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:48.575817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:48.575817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:48.575817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:48.575817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:48.575817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:48.575817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:14:48.575817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:14:48.575817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:14:48.575817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:14:48.576958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:48.576958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:48.576958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:48.576958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:48.576958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:48.576958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:48.576958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:48.576958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:48.576958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:48.576958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:48.576958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:48.576958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:48.576958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:48.576958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:48.576958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:48.576958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:48.576958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:14:48.577970 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:14:48.577970 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:48.577970 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:48.582974 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 231/470] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:14:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:48.583951 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6320ae61-63c1-418f-8d27-9a8e9f1ac3fc None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:48.620942 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:48.661268 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:48.684625 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600fd75c-781a-4c3e-b0fc-79c284c462f6 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:48.712724 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6320ae61-63c1-418f-8d27-9a8e9f1ac3fc None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:48.733853 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea7a51-83fe-4d9d-9b88-9941af389003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:48.734478 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea7a51-83fe-4d9d-9b88-9941af389003 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:48.734821 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea7a51-83fe-4d9d-9b88-9941af389003 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:48.735099 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea7a51-83fe-4d9d-9b88-9941af389003 None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:48.735760 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea7a51-83fe-4d9d-9b88-9941af389003 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:48.736095 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea7a51-83fe-4d9d-9b88-9941af389003 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:48.737923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea7a51-83fe-4d9d-9b88-9941af389003 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:48.750523 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 240/471] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:48.751400 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea7a51-83fe-4d9d-9b88-9941af389003 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:48.762173 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea2365a-b5e1-4ade-84fb-1c4234c88446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:48.763103 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2365a-b5e1-4ade-84fb-1c4234c88446 None admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:48.763595 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2365a-b5e1-4ade-84fb-1c4234c88446 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:48.764060 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2365a-b5e1-4ade-84fb-1c4234c88446 None admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:48.764877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea2365a-b5e1-4ade-84fb-1c4234c88446 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:48.767167 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea2365a-b5e1-4ade-84fb-1c4234c88446 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:48.798151 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 232/472] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Aug 23 04:14:48 2022] POST /identity/v3/users => generated 248 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:14:48.800353 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea2365a-b5e1-4ade-84fb-1c4234c88446 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:48.881918 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea7a51-83fe-4d9d-9b88-9941af389003 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:48.923182 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:48.938688 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da435fa-8fbf-4c80-9181-9a2ced5d7f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:48.939328 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da435fa-8fbf-4c80-9181-9a2ced5d7f61 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:48.939605 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da435fa-8fbf-4c80-9181-9a2ced5d7f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:48.939877 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da435fa-8fbf-4c80-9181-9a2ced5d7f61 None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:48.940567 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da435fa-8fbf-4c80-9181-9a2ced5d7f61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:48.940871 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da435fa-8fbf-4c80-9181-9a2ced5d7f61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:48.942316 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da435fa-8fbf-4c80-9181-9a2ced5d7f61 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:48.962993 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea2365a-b5e1-4ade-84fb-1c4234c88446 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:48.965894 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 241/473] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:48.974006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da435fa-8fbf-4c80-9181-9a2ced5d7f61 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:48.980165 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:49.016768 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb5261-90f2-403a-ba79-361fea995291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:49.017474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb5261-90f2-403a-ba79-361fea995291 None admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:49.017769 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb5261-90f2-403a-ba79-361fea995291 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:49.018018 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb5261-90f2-403a-ba79-361fea995291 None admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:49.018660 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb5261-90f2-403a-ba79-361fea995291 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:49.020189 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb5261-90f2-403a-ba79-361fea995291 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:49.052630 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 233/474] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Aug 23 04:14:48 2022] POST /identity/v3/users => generated 248 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:14:49.054409 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dcfddc-cb03-4cb8-9d03-7748f40198b6 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:49.054703 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dcfddc-cb03-4cb8-9d03-7748f40198b6 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:49.054981 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dcfddc-cb03-4cb8-9d03-7748f40198b6 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:49.056865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 234/475] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:49.065598 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68433621-537c-485a-83ef-9fb0986c754d None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:49.066349 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68433621-537c-485a-83ef-9fb0986c754d None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:49.066824 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68433621-537c-485a-83ef-9fb0986c754d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:49.092280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-68433621-537c-485a-83ef-9fb0986c754d None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:49.093975 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-68433621-537c-485a-83ef-9fb0986c754d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:49.105622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da435fa-8fbf-4c80-9181-9a2ced5d7f61 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:49.163338 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac198d-ccbc-427c-ab42-beaf5c7eda40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:49.164273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac198d-ccbc-427c-ab42-beaf5c7eda40 None admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:49.164699 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac198d-ccbc-427c-ab42-beaf5c7eda40 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:49.165142 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac198d-ccbc-427c-ab42-beaf5c7eda40 None admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:49.165948 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ac198d-ccbc-427c-ab42-beaf5c7eda40 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:49.167568 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ac198d-ccbc-427c-ab42-beaf5c7eda40 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:49.187179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 235/476] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:49.196350 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 242/477] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Aug 23 04:14:48 2022] POST /identity/v3/users => generated 249 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:14:49.502482 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68433621-537c-485a-83ef-9fb0986c754d None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:49.576806 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:49.610030 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68433621-537c-485a-83ef-9fb0986c754d None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:49.646296 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ef7a1-3d4c-4d4a-94bd-13ca18ac17a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:49.646917 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ef7a1-3d4c-4d4a-94bd-13ca18ac17a5 None admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:49.647250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ef7a1-3d4c-4d4a-94bd-13ca18ac17a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:49.647524 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ef7a1-3d4c-4d4a-94bd-13ca18ac17a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:49.648299 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ef7a1-3d4c-4d4a-94bd-13ca18ac17a5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:49.649754 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ef7a1-3d4c-4d4a-94bd-13ca18ac17a5 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:49.658968 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.sql.core [None req-193ef7a1-3d4c-4d4a-94bd-13ca18ac17a5 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 23 04:14:49.658968 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 23 04:14:49.658968 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [parameters: {'id': '2dba1ed6a5a040018afa0be891d47157', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 23 04:14:49.658968 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81933) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 23 04:14:49.675837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-193ef7a1-3d4c-4d4a-94bd-13ca18ac17a5 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 23 04:14:49.675837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:49.675837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 04:14:49.675837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 04:14:49.675837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 04:14:49.675837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 04:14:49.675837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 04:14:49.675837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 04:14:49.675837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 04:14:49.675837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application conn.query(q) Aug 23 04:14:49.675837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 04:14:49.675837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 04:14:49.675837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 04:14:49.675837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application result.read() Aug 23 04:14:49.675837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 04:14:49.676741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 04:14:49.676741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 04:14:49.676741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 04:14:49.676741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 04:14:49.676741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 04:14:49.676741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 04:14:49.676741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 04:14:49.676741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 23 04:14:49.676741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:49.676741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 23 04:14:49.676741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:49.676741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:49.676741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 23 04:14:49.676741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 23 04:14:49.676741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 23 04:14:49.676741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return ref.to_dict() Aug 23 04:14:49.676741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 04:14:49.676741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application next(self.gen) Aug 23 04:14:49.677606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 23 04:14:49.677606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application yield resource Aug 23 04:14:49.677606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 04:14:49.677606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application next(self.gen) Aug 23 04:14:49.677606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 23 04:14:49.677606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self.session.rollback() Aug 23 04:14:49.677606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 23 04:14:49.677606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self.force_reraise() Aug 23 04:14:49.677606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 23 04:14:49.677606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise self.value Aug 23 04:14:49.677606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 23 04:14:49.677606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 23 04:14:49.677606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 23 04:14:49.677606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application session.commit() Aug 23 04:14:49.677606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 23 04:14:49.677606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 23 04:14:49.677606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 23 04:14:49.678487 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self._prepare_impl() Aug 23 04:14:49.678487 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 23 04:14:49.678487 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self.session.flush() Aug 23 04:14:49.678487 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 23 04:14:49.678487 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self._flush(objects) Aug 23 04:14:49.678487 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 23 04:14:49.678487 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 23 04:14:49.678487 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 23 04:14:49.678487 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application compat.raise_( Aug 23 04:14:49.678487 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 04:14:49.678487 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception Aug 23 04:14:49.678487 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 23 04:14:49.678487 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application flush_context.execute() Aug 23 04:14:49.678487 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 23 04:14:49.678487 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rec.execute(self) Aug 23 04:14:49.678487 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 23 04:14:49.678487 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 23 04:14:49.679532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 23 04:14:49.679532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 23 04:14:49.679532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 23 04:14:49.679532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 23 04:14:49.679532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 23 04:14:49.679532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 23 04:14:49.679532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 23 04:14:49.679532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 23 04:14:49.679532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 23 04:14:49.679532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 23 04:14:49.679532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 23 04:14:49.679532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 23 04:14:49.679532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 23 04:14:49.679532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 23 04:14:49.679532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 04:14:49.679532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception Aug 23 04:14:49.680417 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 04:14:49.680417 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 04:14:49.680417 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 04:14:49.680417 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 04:14:49.680417 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 04:14:49.680417 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 04:14:49.680417 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 04:14:49.680417 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application conn.query(q) Aug 23 04:14:49.680417 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 04:14:49.680417 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 04:14:49.680417 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 04:14:49.680417 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application result.read() Aug 23 04:14:49.680417 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 04:14:49.680417 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 04:14:49.680417 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 04:14:49.680417 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 04:14:49.681176 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 04:14:49.681176 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 04:14:49.681176 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 04:14:49.681176 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 04:14:49.681176 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 23 04:14:49.681176 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 23 04:14:49.681176 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [parameters: {'id': '2dba1ed6a5a040018afa0be891d47157', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 23 04:14:49.681176 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 23 04:14:49.681176 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:49.681176 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:14:49.681176 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:49.681176 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:49.681176 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:49.681176 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:49.681176 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:49.681176 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:49.681931 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:49.681931 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:49.681931 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:49.681931 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:49.681931 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:49.681931 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:49.681931 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 23 04:14:49.681931 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 23 04:14:49.681931 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:49.681931 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:49.681931 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 23 04:14:49.681931 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 23 04:14:49.681931 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 23 04:14:49.681931 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 23 04:14:49.681931 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 23 04:14:49.681931 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:49.684400 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 236/478] 10.222.0.18 () {60 vars in 1131 bytes} [Tue Aug 23 04:14:49 2022] POST /identity/v3/roles => generated 144 bytes in 175 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:49.685686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac198d-ccbc-427c-ab42-beaf5c7eda40 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:49.771579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:49.823778 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac198d-ccbc-427c-ab42-beaf5c7eda40 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:49.877540 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf871d-6482-43f1-b0a0-e0afd5a7a14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:49.878197 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf871d-6482-43f1-b0a0-e0afd5a7a14c None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:49.878478 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf871d-6482-43f1-b0a0-e0afd5a7a14c None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:49.878751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf871d-6482-43f1-b0a0-e0afd5a7a14c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:49.889758 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf871d-6482-43f1-b0a0-e0afd5a7a14c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:49.894913 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-e0bf871d-6482-43f1-b0a0-e0afd5a7a14c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 04:14:49.894913 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:49.894913 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:49.894913 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:49.894913 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:49.894913 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:49.894913 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:49.894913 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:49.894913 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:49.894913 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:49.894913 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:49.894913 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:49.894913 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:49.894913 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:49.894913 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:49.894913 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:49.895847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:49.895847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:49.895847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:49.895847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:49.895847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:49.895847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:49.895847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:49.895847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:49.895847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:49.895847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:49.895847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:49.895847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:49.895847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:49.895847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:49.895847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:49.895847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:49.896666 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:49.896666 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:49.896666 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:49.896666 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:49.896666 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:49.896666 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:49.896666 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:49.896666 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:49.896666 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:49.896666 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:49.896666 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 04:14:49.896666 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:49.896666 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 243/479] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Aug 23 04:14:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:49.902219 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ef7a1-3d4c-4d4a-94bd-13ca18ac17a5 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:50.029795 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ef7a1-3d4c-4d4a-94bd-13ca18ac17a5 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:50.071545 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5e05b-7423-4413-804e-72b086e83436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:50.072548 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5e05b-7423-4413-804e-72b086e83436 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:50.073074 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5e05b-7423-4413-804e-72b086e83436 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:50.073549 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5e05b-7423-4413-804e-72b086e83436 None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:50.074418 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5e05b-7423-4413-804e-72b086e83436 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:50.074916 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5e05b-7423-4413-804e-72b086e83436 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:50.076528 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5e05b-7423-4413-804e-72b086e83436 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:50.079034 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:50.090069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 237/480] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:14:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:50.983598 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6872bb-71ba-4045-939d-6baaba68fb47 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:50.984163 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6872bb-71ba-4045-939d-6baaba68fb47 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:50.984163 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6872bb-71ba-4045-939d-6baaba68fb47 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:50.985808 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 244/481] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:50.991737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58d390-df09-4cb4-9993-7c6f7ac032cc None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:50.992247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58d390-df09-4cb4-9993-7c6f7ac032cc None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:50.992325 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58d390-df09-4cb4-9993-7c6f7ac032cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:51.017022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-3e58d390-df09-4cb4-9993-7c6f7ac032cc None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:51.018326 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-3e58d390-df09-4cb4-9993-7c6f7ac032cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:51.100989 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 238/482] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:51.344951 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0caee1-ed13-46c4-a571-4a4ad17126cb None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:51.345294 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0caee1-ed13-46c4-a571-4a4ad17126cb None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:51.345469 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0caee1-ed13-46c4-a571-4a4ad17126cb None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:51.347087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 245/483] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:51.352490 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5877b-9958-4f71-aec8-d7957abf522a None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:51.352782 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5877b-9958-4f71-aec8-d7957abf522a None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:51.353060 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5877b-9958-4f71-aec8-d7957abf522a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:51.376548 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-a3e5877b-9958-4f71-aec8-d7957abf522a None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:51.377767 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a3e5877b-9958-4f71-aec8-d7957abf522a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:51.465373 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 239/484] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:51.493874 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e52f3-20a6-4f08-8592-b1370de400bf None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:51.494168 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e52f3-20a6-4f08-8592-b1370de400bf None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:51.494433 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e52f3-20a6-4f08-8592-b1370de400bf None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:51.495928 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 246/485] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:14:51.501378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9caa2b-38d4-4bea-8813-941de0c44e56 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:51.501673 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9caa2b-38d4-4bea-8813-941de0c44e56 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:51.501946 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9caa2b-38d4-4bea-8813-941de0c44e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:51.527747 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-bf9caa2b-38d4-4bea-8813-941de0c44e56 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:51.528987 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-bf9caa2b-38d4-4bea-8813-941de0c44e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:51.590586 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e52f3-20a6-4f08-8592-b1370de400bf None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:51.598441 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:51.612326 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 240/486] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:51.652012 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae522e6-618e-43c8-a3bd-eeacaf36c116 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:51.652375 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae522e6-618e-43c8-a3bd-eeacaf36c116 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:51.652638 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae522e6-618e-43c8-a3bd-eeacaf36c116 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:51.654242 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 241/487] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:14:51.667616 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988269f-f096-4cb5-9c89-7e4d0bc9e56d None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:51.668195 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988269f-f096-4cb5-9c89-7e4d0bc9e56d None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:51.668667 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988269f-f096-4cb5-9c89-7e4d0bc9e56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:51.702780 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-0988269f-f096-4cb5-9c89-7e4d0bc9e56d None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:51.704422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-0988269f-f096-4cb5-9c89-7e4d0bc9e56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:51.741143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e52f3-20a6-4f08-8592-b1370de400bf None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:51.781395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:51.803077 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d06a69-191b-4310-8fd9-717a9da24147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:51.804133 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d06a69-191b-4310-8fd9-717a9da24147 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:51.804427 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d06a69-191b-4310-8fd9-717a9da24147 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:51.804701 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d06a69-191b-4310-8fd9-717a9da24147 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:51.815840 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d06a69-191b-4310-8fd9-717a9da24147 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:51.821516 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-54d06a69-191b-4310-8fd9-717a9da24147 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:14:51.821516 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:51.821516 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:51.821516 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:51.821516 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:51.821516 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:51.821516 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:51.821516 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:51.821516 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:51.821516 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:51.821516 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:51.821516 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:51.821516 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:51.821516 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:51.821516 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:51.821516 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:51.822392 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:51.822392 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:51.822392 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:51.822392 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:51.822392 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:51.822392 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:51.822392 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:51.822392 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:51.822392 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:51.822392 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:51.822392 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:51.822392 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:51.822392 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:51.822392 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:51.822392 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:51.822392 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:51.823155 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:51.823155 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:51.823155 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:51.823155 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:51.823155 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:51.823155 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:51.823155 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:51.823155 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:51.823155 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:51.823155 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:51.823155 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:14:51.823155 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:51.823155 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 247/488] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Aug 23 04:14:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:51.824333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d06a69-191b-4310-8fd9-717a9da24147 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:51.833861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 242/489] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:51.834874 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988269f-f096-4cb5-9c89-7e4d0bc9e56d None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:51.961831 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d06a69-191b-4310-8fd9-717a9da24147 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:51.966039 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988269f-f096-4cb5-9c89-7e4d0bc9e56d None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:51.983524 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:52.000076 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462943db-e5fd-44bf-af77-9d86690c1039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:52.000799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462943db-e5fd-44bf-af77-9d86690c1039 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:52.001048 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d4221-0647-42a6-8728-48ab39e7c311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:52.001484 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462943db-e5fd-44bf-af77-9d86690c1039 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:52.001484 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462943db-e5fd-44bf-af77-9d86690c1039 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:52.001779 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d4221-0647-42a6-8728-48ab39e7c311 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:52.006154 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d4221-0647-42a6-8728-48ab39e7c311 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:52.006154 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d4221-0647-42a6-8728-48ab39e7c311 None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:52.006154 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d4221-0647-42a6-8728-48ab39e7c311 None admin] RBAC: Adding query filter params (name=service) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:52.006154 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d4221-0647-42a6-8728-48ab39e7c311 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:52.006154 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d4221-0647-42a6-8728-48ab39e7c311 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:52.012510 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462943db-e5fd-44bf-af77-9d86690c1039 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:52.017117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-462943db-e5fd-44bf-af77-9d86690c1039 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:14:52.017117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:52.017117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:52.017117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:52.017117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:52.017117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:52.017117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:52.017117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:52.017117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:52.017117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:52.017117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:52.017117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:52.017117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:52.017117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:52.017117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:52.017117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:52.019341 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:52.019341 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:52.019341 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:52.019341 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:52.019341 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:52.019341 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:52.019341 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:52.019341 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:52.019341 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:52.019341 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:52.019341 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:52.019341 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:52.019341 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:52.019341 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:52.019341 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:52.019341 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:52.021298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:52.021298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:52.021298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:52.021298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:52.021298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:52.021298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:52.021298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:52.021298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:52.021298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:52.021298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:52.021298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:14:52.021298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:52.036729 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 243/490] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Aug 23 04:14:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:52.036729 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d4221-0647-42a6-8728-48ab39e7c311 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:52.037015 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 248/491] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Aug 23 04:14:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:52.037015 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462943db-e5fd-44bf-af77-9d86690c1039 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:52.070275 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:52.121394 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d4221-0647-42a6-8728-48ab39e7c311 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:52.136017 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462943db-e5fd-44bf-af77-9d86690c1039 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:52.156888 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55444e72-8aa8-40f7-8a95-4fb5370da5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:52.157589 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55444e72-8aa8-40f7-8a95-4fb5370da5ec None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:52.157865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55444e72-8aa8-40f7-8a95-4fb5370da5ec None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:52.158165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55444e72-8aa8-40f7-8a95-4fb5370da5ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:52.169889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55444e72-8aa8-40f7-8a95-4fb5370da5ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:52.171735 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b23517-696a-4d38-8e56-c4b647d6fc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:52.172440 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b23517-696a-4d38-8e56-c4b647d6fc95 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:52.172720 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b23517-696a-4d38-8e56-c4b647d6fc95 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:52.172990 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b23517-696a-4d38-8e56-c4b647d6fc95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:52.174965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-55444e72-8aa8-40f7-8a95-4fb5370da5ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:14:52.174965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:52.174965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:52.174965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:52.174965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:52.174965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:52.174965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:52.174965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:52.174965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:52.174965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:52.174965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:52.174965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:52.174965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:52.174965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:52.174965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:52.174965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:52.175799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:52.175799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:52.175799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:52.175799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:52.175799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:52.175799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:52.175799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:52.175799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:52.175799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:52.175799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:52.175799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:52.175799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:52.175799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:52.175799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:52.175799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:52.175799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:52.176598 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:52.176598 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:52.176598 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:52.176598 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:52.176598 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:52.176598 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:52.176598 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:52.176598 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:52.176598 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:52.176598 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:52.176598 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:14:52.176598 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:52.176598 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 244/492] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Aug 23 04:14:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:52.177229 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55444e72-8aa8-40f7-8a95-4fb5370da5ec None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:52.190599 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b23517-696a-4d38-8e56-c4b647d6fc95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:52.192273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b23517-696a-4d38-8e56-c4b647d6fc95 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:52.207211 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-22b23517-696a-4d38-8e56-c4b647d6fc95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:52.207211 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:52.207211 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:52.207211 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:52.207211 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:52.207211 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:52.207211 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:52.207211 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:52.207211 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:52.207211 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:52.207211 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:52.207211 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:52.207211 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:14:52.207211 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:14:52.207211 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:14:52.207211 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:14:52.208151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:52.208151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:52.208151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:52.208151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:52.208151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:52.208151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:52.208151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:52.208151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:52.208151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:52.208151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:52.208151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:52.208151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:52.208151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:52.208151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:52.208151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:52.208151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:52.208151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:14:52.208967 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:14:52.208967 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:52.208967 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:52.211095 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 249/493] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:52.212102 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b23517-696a-4d38-8e56-c4b647d6fc95 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:52.227075 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:52.288186 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55444e72-8aa8-40f7-8a95-4fb5370da5ec None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:52.313834 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:52.324968 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b23517-696a-4d38-8e56-c4b647d6fc95 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:52.325639 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c29bba-7340-4370-976b-3abc20e15d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:52.326251 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c29bba-7340-4370-976b-3abc20e15d78 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:52.326534 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c29bba-7340-4370-976b-3abc20e15d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:52.326794 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c29bba-7340-4370-976b-3abc20e15d78 None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:52.327873 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c29bba-7340-4370-976b-3abc20e15d78 None admin] RBAC: Adding query filter params (name=service) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:52.327873 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c29bba-7340-4370-976b-3abc20e15d78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:52.329151 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c29bba-7340-4370-976b-3abc20e15d78 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:52.344039 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 245/494] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Aug 23 04:14:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:52.344039 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c29bba-7340-4370-976b-3abc20e15d78 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:52.367829 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7781266-e92c-4eb3-98ac-699ee754d605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:52.368519 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7781266-e92c-4eb3-98ac-699ee754d605 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:52.369206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7781266-e92c-4eb3-98ac-699ee754d605 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:52.369490 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7781266-e92c-4eb3-98ac-699ee754d605 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:52.380733 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7781266-e92c-4eb3-98ac-699ee754d605 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:52.386053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-b7781266-e92c-4eb3-98ac-699ee754d605 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:14:52.386053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:52.386053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:52.386053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:52.386053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:52.386053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:52.386053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:52.386053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:52.386053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:52.386053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:52.386053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:52.386053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:52.386053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:52.386053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:52.386053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:52.386053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:52.386930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:52.386930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:52.386930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:52.386930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:52.386930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:52.386930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:52.386930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:52.386930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:52.386930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:52.386930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:52.386930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:52.386930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:52.386930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:52.386930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:52.386930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:52.386930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:52.387944 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:52.387944 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:52.387944 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:52.387944 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:52.387944 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:52.387944 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:52.387944 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:52.387944 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:52.387944 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:52.387944 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:52.387944 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:14:52.387944 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:52.387944 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 250/495] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Aug 23 04:14:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:52.388999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7781266-e92c-4eb3-98ac-699ee754d605 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:52.467001 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c29bba-7340-4370-976b-3abc20e15d78 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:52.468955 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:52.497206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7781266-e92c-4eb3-98ac-699ee754d605 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:52.506106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6121af7b-fcad-42ad-9989-79cbd13ab6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:52.506846 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121af7b-fcad-42ad-9989-79cbd13ab6cf None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:52.507284 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121af7b-fcad-42ad-9989-79cbd13ab6cf None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:52.507357 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121af7b-fcad-42ad-9989-79cbd13ab6cf None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:52.508032 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6121af7b-fcad-42ad-9989-79cbd13ab6cf None admin] RBAC: Adding query filter params (name=service) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:52.508327 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6121af7b-fcad-42ad-9989-79cbd13ab6cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:52.509709 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6121af7b-fcad-42ad-9989-79cbd13ab6cf None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:52.517879 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 246/496] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Aug 23 04:14:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:52.518659 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6121af7b-fcad-42ad-9989-79cbd13ab6cf None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:52.539101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca079f-55aa-4e5e-ade7-0c0fe09b5a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:52.540147 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca079f-55aa-4e5e-ade7-0c0fe09b5a2b None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:52.540611 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca079f-55aa-4e5e-ade7-0c0fe09b5a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:52.541085 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca079f-55aa-4e5e-ade7-0c0fe09b5a2b None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:52.541900 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ca079f-55aa-4e5e-ade7-0c0fe09b5a2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:52.542484 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ca079f-55aa-4e5e-ade7-0c0fe09b5a2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:52.544003 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ca079f-55aa-4e5e-ade7-0c0fe09b5a2b None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:52.558188 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 251/497] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:52.558804 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca079f-55aa-4e5e-ade7-0c0fe09b5a2b None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:52.576641 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:52.630587 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6121af7b-fcad-42ad-9989-79cbd13ab6cf None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:52.664297 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244742c4-d7d4-4637-b8ae-7579933a8011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:52.664916 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244742c4-d7d4-4637-b8ae-7579933a8011 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:52.665193 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244742c4-d7d4-4637-b8ae-7579933a8011 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:52.665463 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244742c4-d7d4-4637-b8ae-7579933a8011 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:52.667500 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca079f-55aa-4e5e-ade7-0c0fe09b5a2b None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:52.675744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244742c4-d7d4-4637-b8ae-7579933a8011 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:52.677223 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244742c4-d7d4-4637-b8ae-7579933a8011 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:52.691094 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-244742c4-d7d4-4637-b8ae-7579933a8011 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:52.691094 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:52.691094 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:52.691094 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:52.691094 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:52.691094 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:52.691094 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:52.691094 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:52.691094 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:52.691094 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:52.691094 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:52.691094 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:52.691094 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:14:52.691094 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:14:52.691094 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:14:52.691094 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:14:52.691961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:52.691961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:52.691961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:52.691961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:52.691961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:52.691961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:52.691961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:52.691961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:52.691961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:52.691961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:52.691961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:52.691961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:52.691961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:52.691961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:52.691961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:52.691961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:52.691961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:14:52.692813 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:14:52.692813 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:52.692813 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:52.692813 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 247/498] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:52.693032 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244742c4-d7d4-4637-b8ae-7579933a8011 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:52.705835 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3e1d0-8cbf-4a71-bc68-b3064807ff98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:52.706444 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3e1d0-8cbf-4a71-bc68-b3064807ff98 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:52.706721 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3e1d0-8cbf-4a71-bc68-b3064807ff98 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:52.706986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3e1d0-8cbf-4a71-bc68-b3064807ff98 None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:52.707656 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3e1d0-8cbf-4a71-bc68-b3064807ff98 None admin] RBAC: Adding query filter params (name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:52.707947 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3e1d0-8cbf-4a71-bc68-b3064807ff98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:52.709364 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3e1d0-8cbf-4a71-bc68-b3064807ff98 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:52.715293 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:52.719144 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 252/499] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Aug 23 04:14:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:52.719877 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3e1d0-8cbf-4a71-bc68-b3064807ff98 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:52.797122 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244742c4-d7d4-4637-b8ae-7579933a8011 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:52.797994 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:52.824591 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3e1d0-8cbf-4a71-bc68-b3064807ff98 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:52.830935 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16199101-fb97-425e-b30c-32c508fd28e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:52.831567 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16199101-fb97-425e-b30c-32c508fd28e9 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:52.831846 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16199101-fb97-425e-b30c-32c508fd28e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:52.832149 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16199101-fb97-425e-b30c-32c508fd28e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:52.843545 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16199101-fb97-425e-b30c-32c508fd28e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:52.845046 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16199101-fb97-425e-b30c-32c508fd28e9 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:52.858229 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b03add-6769-4560-ac9f-ce574501ca68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:52.858859 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b03add-6769-4560-ac9f-ce574501ca68 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:52.859140 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b03add-6769-4560-ac9f-ce574501ca68 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:52.859380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-16199101-fb97-425e-b30c-32c508fd28e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:52.859380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:52.859380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:52.859380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:52.859380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:52.859380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:52.859380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:52.859380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:52.859380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:52.859380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:52.859380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:52.859380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:52.859380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:14:52.859380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:14:52.859380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:14:52.859380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:14:52.860315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:52.860315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:52.860315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:52.860315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:52.860315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:52.860315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:52.860315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:52.860315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:52.860315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:52.860315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:52.860315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:52.860315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:52.860315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:52.860315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:52.860315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:52.860315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:52.860315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:14:52.861136 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:14:52.861136 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:52.861136 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:52.861292 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b03add-6769-4560-ac9f-ce574501ca68 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:52.861357 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 248/500] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:52.861672 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b65cf-665a-4a4a-9317-ece833035eb8 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:52.861922 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b65cf-665a-4a4a-9317-ece833035eb8 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:52.862192 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b65cf-665a-4a4a-9317-ece833035eb8 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:52.863599 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 249/501] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:52.864101 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b65cf-665a-4a4a-9317-ece833035eb8 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:52.872067 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b03add-6769-4560-ac9f-ce574501ca68 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:52.873535 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b03add-6769-4560-ac9f-ce574501ca68 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:52.886616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-23b03add-6769-4560-ac9f-ce574501ca68 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 04:14:52.886616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:52.886616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:52.886616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:52.886616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:52.886616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:52.886616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:52.886616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:52.886616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:52.886616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:52.886616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:52.886616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:52.886616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:14:52.886616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:14:52.886616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:14:52.886616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:14:52.887698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:52.887698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:52.887698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:14:52.887698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:14:52.887698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:14:52.887698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:14:52.887698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:52.887698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:52.887698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:52.887698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:52.887698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:52.887698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:52.887698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:52.887698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:52.887698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:52.887698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:52.887698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:52.887698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:52.888788 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:52.888788 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:52.888788 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:14:52.888788 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:14:52.888788 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:14:52.888788 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:14:52.888788 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:14:52.888788 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:14:52.888788 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 04:14:52.888788 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:52.888788 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 253/502] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Aug 23 04:14:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:52.889345 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b03add-6769-4560-ac9f-ce574501ca68 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:52.939234 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:52.975159 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b65cf-665a-4a4a-9317-ece833035eb8 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:52.989376 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b03add-6769-4560-ac9f-ce574501ca68 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:53.009606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3cfaa9-1da1-4837-bbd0-a84361a2e303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:53.010222 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3cfaa9-1da1-4837-bbd0-a84361a2e303 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:53.010479 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3cfaa9-1da1-4837-bbd0-a84361a2e303 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:53.010745 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3cfaa9-1da1-4837-bbd0-a84361a2e303 None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:53.011395 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3cfaa9-1da1-4837-bbd0-a84361a2e303 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:53.011692 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3cfaa9-1da1-4837-bbd0-a84361a2e303 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:53.013100 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3cfaa9-1da1-4837-bbd0-a84361a2e303 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:53.022530 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210cf7da-a1a4-484c-a117-0c0a73387ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:53.023163 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210cf7da-a1a4-484c-a117-0c0a73387ba6 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:53.023445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210cf7da-a1a4-484c-a117-0c0a73387ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:53.023719 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210cf7da-a1a4-484c-a117-0c0a73387ba6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:53.030143 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 250/503] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:53.030301 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:53.030380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3cfaa9-1da1-4837-bbd0-a84361a2e303 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:53.036242 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210cf7da-a1a4-484c-a117-0c0a73387ba6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:53.037790 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210cf7da-a1a4-484c-a117-0c0a73387ba6 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:53.052645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-210cf7da-a1a4-484c-a117-0c0a73387ba6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:53.052645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:53.052645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:53.052645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:53.052645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:53.052645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:53.052645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:53.052645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:53.052645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:53.052645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:53.052645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:53.052645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:53.052645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:14:53.052645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:14:53.052645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:14:53.052645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:14:53.053731 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:53.053731 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:53.053731 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:53.053731 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:53.053731 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:53.053731 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:53.053731 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:53.053731 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:53.053731 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:53.053731 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:53.053731 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:53.053731 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:53.053731 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:53.053731 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:53.053731 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:53.053731 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:53.053731 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:14:53.054562 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:14:53.054562 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:53.054562 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:53.054562 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 254/504] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:53.055774 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdeb0e1-5dc9-4b2a-8100-ab5c93edf446 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:53.056098 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdeb0e1-5dc9-4b2a-8100-ab5c93edf446 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:53.056376 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdeb0e1-5dc9-4b2a-8100-ab5c93edf446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:53.073955 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-1fdeb0e1-5dc9-4b2a-8100-ab5c93edf446 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:53.075163 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-1fdeb0e1-5dc9-4b2a-8100-ab5c93edf446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:53.132390 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3cfaa9-1da1-4837-bbd0-a84361a2e303 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:53.141746 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 255/505] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:53.142554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdeb0e1-5dc9-4b2a-8100-ab5c93edf446 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:53.169300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:53.178281 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877fc2c8-f0d2-4ad0-8854-0631e3616483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:53.178947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fc2c8-f0d2-4ad0-8854-0631e3616483 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:53.179233 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fc2c8-f0d2-4ad0-8854-0631e3616483 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:53.179505 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fc2c8-f0d2-4ad0-8854-0631e3616483 None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:53.180222 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877fc2c8-f0d2-4ad0-8854-0631e3616483 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:53.180528 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877fc2c8-f0d2-4ad0-8854-0631e3616483 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:53.181892 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877fc2c8-f0d2-4ad0-8854-0631e3616483 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:53.193435 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 251/506] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:53.194318 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877fc2c8-f0d2-4ad0-8854-0631e3616483 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:53.271639 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdeb0e1-5dc9-4b2a-8100-ab5c93edf446 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:53.290220 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:53.306560 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877fc2c8-f0d2-4ad0-8854-0631e3616483 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:53.309201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310c836f-75ef-4f6f-8f4b-e41e1c33dcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:53.309817 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c836f-75ef-4f6f-8f4b-e41e1c33dcce None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:53.310098 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c836f-75ef-4f6f-8f4b-e41e1c33dcce None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:53.310373 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c836f-75ef-4f6f-8f4b-e41e1c33dcce None admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:53.311105 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310c836f-75ef-4f6f-8f4b-e41e1c33dcce None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:53.311409 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310c836f-75ef-4f6f-8f4b-e41e1c33dcce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:53.312884 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310c836f-75ef-4f6f-8f4b-e41e1c33dcce None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:53.341785 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69cfd3-757f-4ca9-b19c-79863171818a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:53.342415 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69cfd3-757f-4ca9-b19c-79863171818a None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:53.342689 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69cfd3-757f-4ca9-b19c-79863171818a None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:53.342954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69cfd3-757f-4ca9-b19c-79863171818a None admin] PATH_INFO: `/v3/users/glance` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:53.350429 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69cfd3-757f-4ca9-b19c-79863171818a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:53.351872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69cfd3-757f-4ca9-b19c-79863171818a None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:53.359357 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 256/507] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Aug 23 04:14:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:53.359357 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310c836f-75ef-4f6f-8f4b-e41e1c33dcce None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:53.364491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-0f69cfd3-757f-4ca9-b19c-79863171818a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 23 04:14:53.364491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:53.364491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:53.364491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:53.364491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:53.364491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:53.364491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:53.364491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:53.364491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:53.364491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:53.364491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:53.364491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:53.364491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:14:53.364491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:14:53.364491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:14:53.364491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:14:53.365371 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:53.365371 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:53.365371 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:14:53.365371 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:14:53.365371 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:14:53.365371 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:14:53.365371 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:53.365371 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:53.365371 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:53.365371 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:53.365371 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:53.365371 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:53.365371 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:53.365371 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:53.365371 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:53.365371 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:53.365371 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:53.365371 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:53.366278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:53.366278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:53.366278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:14:53.366278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:14:53.366278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:14:53.366278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:14:53.366278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:14:53.366278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:14:53.366278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 23 04:14:53.366278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:53.366278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 252/508] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:53.366832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69cfd3-757f-4ca9-b19c-79863171818a None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:53.416227 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:53.468174 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69cfd3-757f-4ca9-b19c-79863171818a None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:53.473715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310c836f-75ef-4f6f-8f4b-e41e1c33dcce None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:53.502023 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863d4cf-a9d8-4c18-8f4e-a982ad7b76fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:53.502633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863d4cf-a9d8-4c18-8f4e-a982ad7b76fa None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:53.502921 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863d4cf-a9d8-4c18-8f4e-a982ad7b76fa None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:53.503210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863d4cf-a9d8-4c18-8f4e-a982ad7b76fa None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:53.508109 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d2934-e1c7-44ba-a121-5efd43f777b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:53.508706 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d2934-e1c7-44ba-a121-5efd43f777b4 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:53.508980 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d2934-e1c7-44ba-a121-5efd43f777b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:53.509249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d2934-e1c7-44ba-a121-5efd43f777b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:53.509886 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d2934-e1c7-44ba-a121-5efd43f777b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:53.510181 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d2934-e1c7-44ba-a121-5efd43f777b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:53.510698 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0863d4cf-a9d8-4c18-8f4e-a982ad7b76fa None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:53.511568 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d2934-e1c7-44ba-a121-5efd43f777b4 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:53.512165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0863d4cf-a9d8-4c18-8f4e-a982ad7b76fa None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:53.523224 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 257/509] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:53.523582 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-0863d4cf-a9d8-4c18-8f4e-a982ad7b76fa None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 04:14:53.523582 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:53.523582 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:53.523582 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:53.523582 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:53.523582 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:53.523582 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:53.523582 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:53.523582 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:53.523582 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:53.523582 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:53.523582 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:53.523582 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:14:53.523582 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:14:53.523582 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:14:53.523582 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:14:53.524464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:53.524464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:53.524464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:14:53.524464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:14:53.524464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:14:53.524464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:14:53.524464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:53.524464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:53.524464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:53.524464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:53.524464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:53.524464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:53.524464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:53.524464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:53.524464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:53.524464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:53.524464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:53.524464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:53.525351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:53.525351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:53.525351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:14:53.525351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:14:53.525351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:14:53.525351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:14:53.525351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:14:53.525351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:14:53.525351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 04:14:53.525351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:53.525842 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d2934-e1c7-44ba-a121-5efd43f777b4 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:53.525917 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 253/510] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:14:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:53.525917 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863d4cf-a9d8-4c18-8f4e-a982ad7b76fa None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:53.565014 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:53.626411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863d4cf-a9d8-4c18-8f4e-a982ad7b76fa None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:53.632378 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d2934-e1c7-44ba-a121-5efd43f777b4 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:53.635120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:53.659871 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e5d31-0fe2-4571-b2fe-4cc0b74292a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:53.660502 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e5d31-0fe2-4571-b2fe-4cc0b74292a5 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:53.660780 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e5d31-0fe2-4571-b2fe-4cc0b74292a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:53.661049 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e5d31-0fe2-4571-b2fe-4cc0b74292a5 None admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:53.661784 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395e5d31-0fe2-4571-b2fe-4cc0b74292a5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:53.662094 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395e5d31-0fe2-4571-b2fe-4cc0b74292a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:53.663696 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395e5d31-0fe2-4571-b2fe-4cc0b74292a5 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:53.665247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121c106-2f84-4f2b-b1ab-335966240b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:53.665898 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121c106-2f84-4f2b-b1ab-335966240b15 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:53.666204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121c106-2f84-4f2b-b1ab-335966240b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:53.666475 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121c106-2f84-4f2b-b1ab-335966240b15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:53.700293 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 254/511] 10.222.0.18 () {56 vars in 1137 bytes} [Tue Aug 23 04:14:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:53.701090 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e5d31-0fe2-4571-b2fe-4cc0b74292a5 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:53.738964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d121c106-2f84-4f2b-b1ab-335966240b15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:53.740473 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d121c106-2f84-4f2b-b1ab-335966240b15 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:53.772515 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:53.806790 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e5d31-0fe2-4571-b2fe-4cc0b74292a5 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:53.839825 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 258/512] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:14:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:14:53.840704 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121c106-2f84-4f2b-b1ab-335966240b15 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:53.841636 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f249b-d542-42c2-8f39-73d1ca393315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:53.842267 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f249b-d542-42c2-8f39-73d1ca393315 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:53.842543 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f249b-d542-42c2-8f39-73d1ca393315 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:53.842920 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f249b-d542-42c2-8f39-73d1ca393315 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:53.854128 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9f249b-d542-42c2-8f39-73d1ca393315 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:53.855567 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9f249b-d542-42c2-8f39-73d1ca393315 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:53.871438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-5b9f249b-d542-42c2-8f39-73d1ca393315 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:53.871438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:53.871438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:53.871438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:53.871438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:53.871438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:53.871438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:53.871438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:53.871438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:53.871438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:53.871438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:53.871438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:53.871438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:14:53.871438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:14:53.871438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:14:53.871438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:14:53.872384 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:53.872384 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:53.872384 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:53.872384 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:53.872384 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:53.872384 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:53.872384 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:53.872384 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:53.872384 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:53.872384 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:53.872384 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:53.872384 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:53.872384 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:53.872384 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:53.872384 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:53.872384 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:53.872384 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:14:53.873203 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:14:53.873203 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:53.873203 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:53.873578 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 255/513] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:14:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:53.874578 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f249b-d542-42c2-8f39-73d1ca393315 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:53.893879 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:53.939735 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121c106-2f84-4f2b-b1ab-335966240b15 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:53.973712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60992d-ad50-4a0e-9543-a3380496c46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:53.974345 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60992d-ad50-4a0e-9543-a3380496c46b None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:53.974621 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60992d-ad50-4a0e-9543-a3380496c46b None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:53.974889 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60992d-ad50-4a0e-9543-a3380496c46b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:53.976981 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f249b-d542-42c2-8f39-73d1ca393315 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:53.977624 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:53.984304 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60992d-ad50-4a0e-9543-a3380496c46b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:53.985774 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60992d-ad50-4a0e-9543-a3380496c46b None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:53.996753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-be60992d-ad50-4a0e-9543-a3380496c46b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 04:14:53.996753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:53.996753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:53.996753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:53.996753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:53.996753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:53.996753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:53.996753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:53.996753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:53.996753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:53.996753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:53.996753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:53.996753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:14:53.996753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:14:53.996753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:14:53.996753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:14:53.997717 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:53.997717 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:53.997717 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:14:53.997717 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:14:53.997717 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:14:53.997717 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:14:53.997717 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:53.997717 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:53.997717 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:53.997717 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:53.997717 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:53.997717 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:53.997717 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:53.997717 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:53.997717 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:53.997717 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:53.997717 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:53.997717 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:53.998759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:53.998759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:53.998759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:14:53.998759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:14:53.998759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:14:53.998759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:14:53.998759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:14:53.998759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:14:53.998759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 04:14:53.998759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:53.998759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 259/514] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Aug 23 04:14:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:53.999878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60992d-ad50-4a0e-9543-a3380496c46b None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:54.015395 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b11d4-73ad-4044-b159-bf81acb840bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:54.016047 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b11d4-73ad-4044-b159-bf81acb840bd None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:54.016325 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b11d4-73ad-4044-b159-bf81acb840bd None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:54.016597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b11d4-73ad-4044-b159-bf81acb840bd None admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:54.017336 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b11d4-73ad-4044-b159-bf81acb840bd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:54.017636 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b11d4-73ad-4044-b159-bf81acb840bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:54.019073 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b11d4-73ad-4044-b159-bf81acb840bd None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:54.036722 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 256/515] 10.222.0.18 () {56 vars in 1137 bytes} [Tue Aug 23 04:14:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:54.037918 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b11d4-73ad-4044-b159-bf81acb840bd None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:54.099551 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60992d-ad50-4a0e-9543-a3380496c46b None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:54.115955 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:54.132986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77a5c0-5832-4825-9ebf-92c2e5aad85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:54.133596 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77a5c0-5832-4825-9ebf-92c2e5aad85d None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:54.133874 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77a5c0-5832-4825-9ebf-92c2e5aad85d None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:54.134147 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77a5c0-5832-4825-9ebf-92c2e5aad85d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:54.139584 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b11d4-73ad-4044-b159-bf81acb840bd None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:54.173176 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b0c32-32cf-4f69-9086-45436b9f8264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:54.173896 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b0c32-32cf-4f69-9086-45436b9f8264 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:54.174170 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b0c32-32cf-4f69-9086-45436b9f8264 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:54.174434 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b0c32-32cf-4f69-9086-45436b9f8264 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:54.184575 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b0c32-32cf-4f69-9086-45436b9f8264 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:54.186035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b0c32-32cf-4f69-9086-45436b9f8264 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:54.193640 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:54.200632 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-c62b0c32-32cf-4f69-9086-45436b9f8264 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:54.200632 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:54.200632 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:54.200632 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:54.200632 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:54.200632 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:54.200632 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:54.200632 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:54.200632 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:54.200632 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:54.200632 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:54.200632 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:54.200632 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:14:54.200632 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:14:54.200632 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:14:54.200632 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:14:54.201669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:54.201669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:54.201669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:54.201669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:54.201669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:54.201669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:54.201669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:54.201669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:54.201669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:54.201669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:54.201669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:54.201669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:54.201669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:54.201669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:54.201669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:54.201669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:54.201669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:14:54.202683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:14:54.202683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:54.202683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:54.202683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 257/516] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:14:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:54.202910 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b0c32-32cf-4f69-9086-45436b9f8264 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:54.204442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de77a5c0-5832-4825-9ebf-92c2e5aad85d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:54.205931 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de77a5c0-5832-4825-9ebf-92c2e5aad85d None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:54.301152 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b0c32-32cf-4f69-9086-45436b9f8264 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:54.304680 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 260/517] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:14:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:14:54.305513 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77a5c0-5832-4825-9ebf-92c2e5aad85d None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:54.329868 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:54.338118 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132945a9-e649-40dd-8dd8-61113f79ffca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:54.339101 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132945a9-e649-40dd-8dd8-61113f79ffca None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:54.339376 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132945a9-e649-40dd-8dd8-61113f79ffca None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:54.339676 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132945a9-e649-40dd-8dd8-61113f79ffca None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:54.340344 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132945a9-e649-40dd-8dd8-61113f79ffca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:54.340648 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132945a9-e649-40dd-8dd8-61113f79ffca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:54.342029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132945a9-e649-40dd-8dd8-61113f79ffca None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:54.353674 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 258/518] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:54.354535 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132945a9-e649-40dd-8dd8-61113f79ffca None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:54.403955 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:54.411271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77a5c0-5832-4825-9ebf-92c2e5aad85d None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:54.447747 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b787f95-516b-4c81-9c47-6e8b6e868603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:54.448368 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b787f95-516b-4c81-9c47-6e8b6e868603 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:54.448640 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b787f95-516b-4c81-9c47-6e8b6e868603 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:54.448923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b787f95-516b-4c81-9c47-6e8b6e868603 None admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:54.449645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b787f95-516b-4c81-9c47-6e8b6e868603 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:54.449927 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b787f95-516b-4c81-9c47-6e8b6e868603 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:54.451376 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b787f95-516b-4c81-9c47-6e8b6e868603 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:54.460948 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132945a9-e649-40dd-8dd8-61113f79ffca None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:54.472547 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 261/519] 10.222.0.18 () {56 vars in 1139 bytes} [Tue Aug 23 04:14:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:54.473238 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b787f95-516b-4c81-9c47-6e8b6e868603 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:54.494139 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca21a23-e49d-415c-8b5e-704ac3db4aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:54.494750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca21a23-e49d-415c-8b5e-704ac3db4aac None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:54.495025 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca21a23-e49d-415c-8b5e-704ac3db4aac None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:54.495304 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca21a23-e49d-415c-8b5e-704ac3db4aac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:54.550569 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:54.563991 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca21a23-e49d-415c-8b5e-704ac3db4aac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:54.565625 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca21a23-e49d-415c-8b5e-704ac3db4aac None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:54.570331 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b787f95-516b-4c81-9c47-6e8b6e868603 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:54.611738 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf35686-0e6b-47f2-9561-715a8aa4bf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:54.612440 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf35686-0e6b-47f2-9561-715a8aa4bf14 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:54.612718 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf35686-0e6b-47f2-9561-715a8aa4bf14 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:54.612976 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf35686-0e6b-47f2-9561-715a8aa4bf14 None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:54.613841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf35686-0e6b-47f2-9561-715a8aa4bf14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:54.614147 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf35686-0e6b-47f2-9561-715a8aa4bf14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:54.615901 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf35686-0e6b-47f2-9561-715a8aa4bf14 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:54.631939 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 262/520] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:54.633195 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf35686-0e6b-47f2-9561-715a8aa4bf14 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:54.648741 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:54.672408 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 259/521] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:14:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:14:54.673228 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca21a23-e49d-415c-8b5e-704ac3db4aac None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:54.736577 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf35686-0e6b-47f2-9561-715a8aa4bf14 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:54.759415 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:54.770330 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01dc90b-0d86-43b0-8480-1d72bc608f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:54.771058 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dc90b-0d86-43b0-8480-1d72bc608f5b None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:54.771404 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dc90b-0d86-43b0-8480-1d72bc608f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:54.771745 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dc90b-0d86-43b0-8480-1d72bc608f5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:54.772112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca21a23-e49d-415c-8b5e-704ac3db4aac None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:54.782955 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01dc90b-0d86-43b0-8480-1d72bc608f5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:54.784454 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01dc90b-0d86-43b0-8480-1d72bc608f5b None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:54.798465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-d01dc90b-0d86-43b0-8480-1d72bc608f5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:54.798465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:54.798465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:54.798465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:54.798465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:54.798465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:54.798465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:54.798465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:54.798465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:54.798465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:54.798465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:54.798465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:54.798465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:14:54.798465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:14:54.798465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:14:54.798465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:14:54.799322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:54.799322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:54.799322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:54.799322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:54.799322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:54.799322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:54.799322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:54.799322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:54.799322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:54.799322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:54.799322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:54.799322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:54.799322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:54.799322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:54.799322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:54.799322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:54.799322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:14:54.800193 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:14:54.800193 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:54.800193 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:54.800193 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 263/522] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:14:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:54.800398 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01dc90b-0d86-43b0-8480-1d72bc608f5b None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:54.810148 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df7ead-b7e6-46b7-970b-2413b222de5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:54.810765 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df7ead-b7e6-46b7-970b-2413b222de5f None admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:54.811045 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df7ead-b7e6-46b7-970b-2413b222de5f None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:54.811320 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df7ead-b7e6-46b7-970b-2413b222de5f None admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:54.811968 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df7ead-b7e6-46b7-970b-2413b222de5f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:54.813410 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df7ead-b7e6-46b7-970b-2413b222de5f None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:54.837038 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 260/523] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Aug 23 04:14:54 2022] POST /identity/v3/users => generated 247 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:14:54.837928 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df7ead-b7e6-46b7-970b-2413b222de5f None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:54.869401 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:54.917727 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01dc90b-0d86-43b0-8480-1d72bc608f5b None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:54.956966 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065411cf-ed43-4add-96dd-c61951624257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:54.957475 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df7ead-b7e6-46b7-970b-2413b222de5f None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:54.957694 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065411cf-ed43-4add-96dd-c61951624257 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:54.957967 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065411cf-ed43-4add-96dd-c61951624257 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:54.958234 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065411cf-ed43-4add-96dd-c61951624257 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:54.998388 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5a8ca-c90e-4a5f-ac83-272f3bbaa546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:54.999351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5a8ca-c90e-4a5f-ac83-272f3bbaa546 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:54.999351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5a8ca-c90e-4a5f-ac83-272f3bbaa546 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:54.999615 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5a8ca-c90e-4a5f-ac83-272f3bbaa546 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:55.008443 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df5a8ca-c90e-4a5f-ac83-272f3bbaa546 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:55.009898 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df5a8ca-c90e-4a5f-ac83-272f3bbaa546 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:55.025064 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-5df5a8ca-c90e-4a5f-ac83-272f3bbaa546 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:14:55.025064 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:55.025064 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:55.025064 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:55.025064 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:55.025064 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:55.025064 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:55.025064 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:55.025064 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:55.025064 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:55.025064 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:55.025064 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:55.025064 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:14:55.025064 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:14:55.025064 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:14:55.026350 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:14:55.026350 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:55.026350 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:55.026350 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:55.026350 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:55.026350 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:55.026350 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:55.026350 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:55.026350 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:55.026350 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:55.026350 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:55.026350 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:55.026350 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:55.026350 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:55.026350 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:55.026350 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:55.026350 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:55.027286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:14:55.027286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:14:55.027286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:14:55.027286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:14:55.027286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:14:55.027286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:14:55.027286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:14:55.027286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:55.027286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 261/524] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:14:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:55.027286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5a8ca-c90e-4a5f-ac83-272f3bbaa546 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:55.032835 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:55.043181 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065411cf-ed43-4add-96dd-c61951624257 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:55.044713 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065411cf-ed43-4add-96dd-c61951624257 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:55.112199 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:55.149551 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5a8ca-c90e-4a5f-ac83-272f3bbaa546 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:55.155545 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 264/525] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:14:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:14:55.155545 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065411cf-ed43-4add-96dd-c61951624257 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:55.191437 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f870ef-9344-481d-b887-ef798797b248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:55.192355 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f870ef-9344-481d-b887-ef798797b248 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:55.192800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f870ef-9344-481d-b887-ef798797b248 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:55.193210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f870ef-9344-481d-b887-ef798797b248 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:55.206142 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f870ef-9344-481d-b887-ef798797b248 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:55.207968 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f870ef-9344-481d-b887-ef798797b248 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:55.225196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-e0f870ef-9344-481d-b887-ef798797b248 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:55.225196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:55.225196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:55.225196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:55.225196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:55.225196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:55.225196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:55.225196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:55.225196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:55.225196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:55.225196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:55.225196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:55.225196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:14:55.225196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:14:55.225196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:14:55.225196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:14:55.226954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:55.226954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:55.226954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:55.226954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:55.226954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:55.226954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:55.226954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:55.226954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:55.226954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:55.226954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:55.226954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:55.226954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:55.226954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:55.226954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:55.226954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:55.226954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:55.226954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:14:55.228089 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:14:55.228089 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:55.228089 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:55.234160 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 262/526] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:14:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:55.235160 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f870ef-9344-481d-b887-ef798797b248 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:55.259399 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065411cf-ed43-4add-96dd-c61951624257 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:55.297754 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29635dbd-406f-4780-9415-d30cf78e548b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:55.298350 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29635dbd-406f-4780-9415-d30cf78e548b None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:55.298642 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29635dbd-406f-4780-9415-d30cf78e548b None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:55.298919 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29635dbd-406f-4780-9415-d30cf78e548b None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:55.299573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29635dbd-406f-4780-9415-d30cf78e548b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:55.299877 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29635dbd-406f-4780-9415-d30cf78e548b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:55.300766 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:55.301608 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29635dbd-406f-4780-9415-d30cf78e548b None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:55.315962 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 265/527] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:14:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:55.315962 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29635dbd-406f-4780-9415-d30cf78e548b None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:55.342212 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:55.358605 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f870ef-9344-481d-b887-ef798797b248 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:55.394556 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b835be-4362-422d-b1e6-1a3bcb396f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:55.395167 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b835be-4362-422d-b1e6-1a3bcb396f77 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:55.395437 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b835be-4362-422d-b1e6-1a3bcb396f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:55.395715 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b835be-4362-422d-b1e6-1a3bcb396f77 None admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:55.396447 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b835be-4362-422d-b1e6-1a3bcb396f77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:55.396750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b835be-4362-422d-b1e6-1a3bcb396f77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:55.398834 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b835be-4362-422d-b1e6-1a3bcb396f77 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:55.417510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 263/528] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:14:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:55.418270 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b835be-4362-422d-b1e6-1a3bcb396f77 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:55.427815 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29635dbd-406f-4780-9415-d30cf78e548b None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:55.467101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59d64b-1a5d-4f47-9512-3ac3c442f657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:55.467734 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59d64b-1a5d-4f47-9512-3ac3c442f657 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:55.468033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59d64b-1a5d-4f47-9512-3ac3c442f657 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:55.468301 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59d64b-1a5d-4f47-9512-3ac3c442f657 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:55.479495 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59d64b-1a5d-4f47-9512-3ac3c442f657 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:55.481025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59d64b-1a5d-4f47-9512-3ac3c442f657 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:55.495694 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-6d59d64b-1a5d-4f47-9512-3ac3c442f657 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:55.495694 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:55.495694 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:55.495694 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:55.495694 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:55.495694 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:55.495694 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:55.495694 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:55.495694 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:55.495694 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:55.495694 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:55.495694 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:55.495694 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:14:55.495694 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:14:55.495694 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:14:55.495694 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:14:55.496862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:55.496862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:55.496862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:55.496862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:55.496862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:55.496862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:55.496862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:55.496862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:55.496862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:55.496862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:55.496862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:55.496862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:55.496862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:55.496862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:55.496862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:55.496862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:55.496862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:14:55.497676 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:14:55.497676 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:55.497676 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:55.497676 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 266/529] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:14:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:55.498637 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59d64b-1a5d-4f47-9512-3ac3c442f657 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:55.543162 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b835be-4362-422d-b1e6-1a3bcb396f77 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:55.560904 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:55.578107 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:55.581759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32acaaa-d483-4d5e-8c98-7429791a0ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:55.582376 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32acaaa-d483-4d5e-8c98-7429791a0ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:55.582657 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32acaaa-d483-4d5e-8c98-7429791a0ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:55.582918 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32acaaa-d483-4d5e-8c98-7429791a0ad9 None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:55.583564 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32acaaa-d483-4d5e-8c98-7429791a0ad9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:55.583848 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32acaaa-d483-4d5e-8c98-7429791a0ad9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:55.585370 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32acaaa-d483-4d5e-8c98-7429791a0ad9 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:55.596888 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 264/530] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:14:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:55.602125 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32acaaa-d483-4d5e-8c98-7429791a0ad9 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:55.605481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59d64b-1a5d-4f47-9512-3ac3c442f657 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:55.643018 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e72030-c2a9-4997-b7cf-9653e9bb16b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:55.643653 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e72030-c2a9-4997-b7cf-9653e9bb16b9 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:55.644310 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e72030-c2a9-4997-b7cf-9653e9bb16b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:55.644599 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e72030-c2a9-4997-b7cf-9653e9bb16b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:55.652129 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e72030-c2a9-4997-b7cf-9653e9bb16b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:55.653611 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e72030-c2a9-4997-b7cf-9653e9bb16b9 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:55.666346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-c8e72030-c2a9-4997-b7cf-9653e9bb16b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:14:55.666346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:55.666346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:55.666346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:55.666346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:55.666346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:55.666346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:55.666346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:55.666346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:55.666346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:55.666346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:55.666346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:55.666346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:14:55.666346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:14:55.666346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:14:55.667161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:14:55.667161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:55.667161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:55.667161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:55.667161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:55.667161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:55.667161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:55.667161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:55.667161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:55.667161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:55.667161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:55.667161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:55.667161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:55.667161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:55.667161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:55.667161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:55.667161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:55.668426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:14:55.668426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:14:55.668426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:14:55.668426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:14:55.668426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:14:55.668426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:14:55.668426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:14:55.668426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:55.668426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 267/531] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:14:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:55.668426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e72030-c2a9-4997-b7cf-9653e9bb16b9 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:55.707212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32acaaa-d483-4d5e-8c98-7429791a0ad9 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:55.742794 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a30a8-5deb-4efb-a786-e93d0558e88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:55.743492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a30a8-5deb-4efb-a786-e93d0558e88a None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:55.743774 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a30a8-5deb-4efb-a786-e93d0558e88a None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:55.744074 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a30a8-5deb-4efb-a786-e93d0558e88a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:55.744803 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92a30a8-5deb-4efb-a786-e93d0558e88a None admin] RBAC: Adding query filter params (role.id=bdf71ef6ab10430e8c356cd4d952ff5c, scope.project.id=9f4373dfb5f9425eb73669e24d18a08f, user.id=cd7b11f479a54adc8d9b8641859b07a1) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:55.745107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92a30a8-5deb-4efb-a786-e93d0558e88a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:55.746538 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92a30a8-5deb-4efb-a786-e93d0558e88a None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:55.763943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 265/532] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:14:55 2022] GET /identity/v3/role_assignments?user.id=cd7b11f479a54adc8d9b8641859b07a1&scope.project.id=9f4373dfb5f9425eb73669e24d18a08f&role.id=bdf71ef6ab10430e8c356cd4d952ff5c => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:55.763943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a30a8-5deb-4efb-a786-e93d0558e88a None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:55.797703 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e72030-c2a9-4997-b7cf-9653e9bb16b9 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:55.813013 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:55.841051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ca3fe-95d8-4c31-860c-07b82845f70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:55.841820 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ca3fe-95d8-4c31-860c-07b82845f70d None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:55.842198 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ca3fe-95d8-4c31-860c-07b82845f70d None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:55.842535 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ca3fe-95d8-4c31-860c-07b82845f70d None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:55.843279 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ca3fe-95d8-4c31-860c-07b82845f70d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:55.843665 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ca3fe-95d8-4c31-860c-07b82845f70d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:55.845231 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ca3fe-95d8-4c31-860c-07b82845f70d None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:55.848171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:55.859582 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 268/533] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:14:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:55.860578 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ca3fe-95d8-4c31-860c-07b82845f70d None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:55.896038 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a30a8-5deb-4efb-a786-e93d0558e88a None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:55.935229 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48780b53-5bc3-4777-8935-630e9fd8717a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:55.936089 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48780b53-5bc3-4777-8935-630e9fd8717a None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:55.936368 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48780b53-5bc3-4777-8935-630e9fd8717a None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:55.936635 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48780b53-5bc3-4777-8935-630e9fd8717a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:55.945593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48780b53-5bc3-4777-8935-630e9fd8717a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:55.947949 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48780b53-5bc3-4777-8935-630e9fd8717a None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:55.964273 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-48780b53-5bc3-4777-8935-630e9fd8717a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:14:55.964273 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:55.964273 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:55.964273 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:55.964273 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:55.964273 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:55.964273 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:55.964273 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:55.964273 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:55.964273 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:55.964273 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:55.964273 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:55.964273 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:14:55.964273 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:14:55.964273 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:14:55.965214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:14:55.965214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:55.965214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:55.965214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:55.965214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:55.965214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:55.965214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:55.965214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:55.965214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:55.965214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:55.965214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:55.965214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:55.965214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:55.965214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:55.965214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:55.965214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:55.965214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:55.966476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:14:55.966476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:14:55.966476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:14:55.966476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:14:55.966476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:14:55.966476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:14:55.966476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:14:55.966476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:55.966476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 266/534] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:14:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:55.967161 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48780b53-5bc3-4777-8935-630e9fd8717a None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:55.977332 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ca3fe-95d8-4c31-860c-07b82845f70d None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:56.020581 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6e390-00aa-4f60-ac4e-e46018574c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:56.021204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6e390-00aa-4f60-ac4e-e46018574c3b None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:56.021571 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6e390-00aa-4f60-ac4e-e46018574c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:56.021945 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6e390-00aa-4f60-ac4e-e46018574c3b None admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:56.022728 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6e390-00aa-4f60-ac4e-e46018574c3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:56.023038 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6e390-00aa-4f60-ac4e-e46018574c3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:56.024507 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6e390-00aa-4f60-ac4e-e46018574c3b None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:56.043894 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 269/535] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:14:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:56.044794 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6e390-00aa-4f60-ac4e-e46018574c3b None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:56.079066 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:56.081963 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48780b53-5bc3-4777-8935-630e9fd8717a None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:56.118215 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88472f-e492-476c-95bf-c48dde3f43af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:56.118869 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88472f-e492-476c-95bf-c48dde3f43af None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:56.119005 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:56.119426 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88472f-e492-476c-95bf-c48dde3f43af None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:56.119426 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88472f-e492-476c-95bf-c48dde3f43af None admin] PATH_INFO: `/v3/projects/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:56.126730 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88472f-e492-476c-95bf-c48dde3f43af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:56.128202 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88472f-e492-476c-95bf-c48dde3f43af None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:56.139683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-ac88472f-e492-476c-95bf-c48dde3f43af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:14:56.139683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:56.139683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:56.139683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:56.139683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:56.139683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:56.139683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:56.139683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:56.139683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:56.139683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:56.139683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:56.139683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:56.139683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:14:56.139683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:14:56.139683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:14:56.140737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:14:56.140737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:56.140737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:56.140737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:56.140737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:56.140737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:56.140737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:56.140737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:56.140737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:56.140737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:56.140737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:56.140737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:56.140737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:56.140737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:56.140737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:56.140737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:56.140737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:56.141745 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:14:56.141745 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:14:56.141745 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:14:56.141745 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:14:56.141745 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:14:56.141745 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:14:56.141745 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:14:56.141745 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:56.145847 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 267/536] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:14:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:56.146985 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88472f-e492-476c-95bf-c48dde3f43af None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:56.159394 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6e390-00aa-4f60-ac4e-e46018574c3b None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:56.196891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcc5b6-163d-4d1f-ac77-1f6e7cd650bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:56.197820 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcc5b6-163d-4d1f-ac77-1f6e7cd650bc None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:56.198232 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcc5b6-163d-4d1f-ac77-1f6e7cd650bc None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:56.198638 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcc5b6-163d-4d1f-ac77-1f6e7cd650bc None admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:56.199477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bcc5b6-163d-4d1f-ac77-1f6e7cd650bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:56.199928 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bcc5b6-163d-4d1f-ac77-1f6e7cd650bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:56.201741 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bcc5b6-163d-4d1f-ac77-1f6e7cd650bc None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:56.215865 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 270/537] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:14:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:56.217031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcc5b6-163d-4d1f-ac77-1f6e7cd650bc None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:56.255412 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88472f-e492-476c-95bf-c48dde3f43af None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:56.289469 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8632aecf-7ac8-43df-9d07-7aa7734a78bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:56.290077 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8632aecf-7ac8-43df-9d07-7aa7734a78bc None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:56.290352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8632aecf-7ac8-43df-9d07-7aa7734a78bc None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:56.290622 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8632aecf-7ac8-43df-9d07-7aa7734a78bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:56.291358 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8632aecf-7ac8-43df-9d07-7aa7734a78bc None admin] RBAC: Adding query filter params (role.id=bdf71ef6ab10430e8c356cd4d952ff5c, scope.project.id=9f4373dfb5f9425eb73669e24d18a08f, user.id=4f1bf93cff6c46e0801bb827203cb5ad) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:56.291657 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8632aecf-7ac8-43df-9d07-7aa7734a78bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:56.293136 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8632aecf-7ac8-43df-9d07-7aa7734a78bc None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:56.308301 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 268/538] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:14:56 2022] GET /identity/v3/role_assignments?user.id=4f1bf93cff6c46e0801bb827203cb5ad&scope.project.id=9f4373dfb5f9425eb73669e24d18a08f&role.id=bdf71ef6ab10430e8c356cd4d952ff5c => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:56.309315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8632aecf-7ac8-43df-9d07-7aa7734a78bc None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:56.330111 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcc5b6-163d-4d1f-ac77-1f6e7cd650bc None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:56.349850 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:56.358979 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:56.374548 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21683b48-ed4a-4015-a82b-36586587f15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:56.375571 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21683b48-ed4a-4015-a82b-36586587f15c None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:56.376063 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21683b48-ed4a-4015-a82b-36586587f15c None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:56.376549 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21683b48-ed4a-4015-a82b-36586587f15c None admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:56.377424 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21683b48-ed4a-4015-a82b-36586587f15c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:56.377912 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21683b48-ed4a-4015-a82b-36586587f15c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:56.379658 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21683b48-ed4a-4015-a82b-36586587f15c None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:56.398064 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 271/539] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:14:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:56.404664 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21683b48-ed4a-4015-a82b-36586587f15c None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:56.475869 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8632aecf-7ac8-43df-9d07-7aa7734a78bc None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:56.515941 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec27887-731e-45b7-9344-597fa3af05df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:56.515941 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec27887-731e-45b7-9344-597fa3af05df None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:56.515941 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec27887-731e-45b7-9344-597fa3af05df None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:56.515941 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec27887-731e-45b7-9344-597fa3af05df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:56.516361 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec27887-731e-45b7-9344-597fa3af05df None admin] RBAC: Adding query filter params (role.id=bdf71ef6ab10430e8c356cd4d952ff5c, scope.project.id=9f4373dfb5f9425eb73669e24d18a08f, user.id=2db57fe3915446acb57b4f84ca353379) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:56.516361 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec27887-731e-45b7-9344-597fa3af05df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:56.516361 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec27887-731e-45b7-9344-597fa3af05df None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:56.521530 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 269/540] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:14:56 2022] GET /identity/v3/role_assignments?user.id=2db57fe3915446acb57b4f84ca353379&scope.project.id=9f4373dfb5f9425eb73669e24d18a08f&role.id=bdf71ef6ab10430e8c356cd4d952ff5c => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:56.580178 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21683b48-ed4a-4015-a82b-36586587f15c None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:56.620785 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e6f15-98f6-4709-a072-d89ff3517b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:56.621867 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e6f15-98f6-4709-a072-d89ff3517b2f None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:56.622363 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e6f15-98f6-4709-a072-d89ff3517b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:56.622870 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e6f15-98f6-4709-a072-d89ff3517b2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:56.623879 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e6f15-98f6-4709-a072-d89ff3517b2f None admin] RBAC: Adding query filter params (role.id=bdf71ef6ab10430e8c356cd4d952ff5c, scope.project.id=9f4373dfb5f9425eb73669e24d18a08f, user.id=fdca9c48e5d740f6bda8c48072b54c0b) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:56.624456 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e6f15-98f6-4709-a072-d89ff3517b2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:56.626126 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e6f15-98f6-4709-a072-d89ff3517b2f None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:56.633585 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 272/541] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:14:56 2022] GET /identity/v3/role_assignments?user.id=fdca9c48e5d740f6bda8c48072b54c0b&scope.project.id=9f4373dfb5f9425eb73669e24d18a08f&role.id=bdf71ef6ab10430e8c356cd4d952ff5c => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:14:56.835253 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de3eb5-98eb-4fd9-ab22-0a0bfe7de69b None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:56.835465 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de3eb5-98eb-4fd9-ab22-0a0bfe7de69b None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:56.835732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de3eb5-98eb-4fd9-ab22-0a0bfe7de69b None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:56.837285 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 270/542] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:56.843622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44131b54-9af8-426a-a377-373a70a9dfc1 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:56.843924 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44131b54-9af8-426a-a377-373a70a9dfc1 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:56.844247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44131b54-9af8-426a-a377-373a70a9dfc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:56.873012 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-44131b54-9af8-426a-a377-373a70a9dfc1 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:56.874256 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-44131b54-9af8-426a-a377-373a70a9dfc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:56.923507 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:56.970760 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 273/543] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:57.318532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de3eb5-98eb-4fd9-ab22-0a0bfe7de69b None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:57.427084 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:57.440282 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de3eb5-98eb-4fd9-ab22-0a0bfe7de69b None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:57.475698 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e23691-7a4a-4ba7-93d6-ea6e048130a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:57.476367 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e23691-7a4a-4ba7-93d6-ea6e048130a2 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:57.476640 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e23691-7a4a-4ba7-93d6-ea6e048130a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:57.476905 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e23691-7a4a-4ba7-93d6-ea6e048130a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:57.487986 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e23691-7a4a-4ba7-93d6-ea6e048130a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:57.492593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-d1e23691-7a4a-4ba7-93d6-ea6e048130a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:14:57.492593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:57.492593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:57.492593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:57.492593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:57.492593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:57.492593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:57.492593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:57.492593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:57.492593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:57.492593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:57.492593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:57.492593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:57.492593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:57.492593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:57.492593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:57.493444 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:57.493444 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:57.493444 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:57.493444 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:57.493444 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:57.493444 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:57.493444 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:57.493444 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:57.493444 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:57.493444 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:57.493444 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:57.493444 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:57.493444 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:57.493444 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:57.493444 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:57.493444 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:57.494796 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:57.494796 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:57.494796 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:57.494796 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:57.494796 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:57.494796 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:57.494796 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:57.494796 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:57.494796 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:57.494796 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:57.494796 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:14:57.494796 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:57.494796 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 271/544] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Aug 23 04:14:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:57.499570 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44131b54-9af8-426a-a377-373a70a9dfc1 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:57.654028 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44131b54-9af8-426a-a377-373a70a9dfc1 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:57.695479 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d68307-9ffd-43e6-8677-f23f196ff7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:57.696128 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d68307-9ffd-43e6-8677-f23f196ff7c3 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:57.696410 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d68307-9ffd-43e6-8677-f23f196ff7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:57.696682 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d68307-9ffd-43e6-8677-f23f196ff7c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:57.697360 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d68307-9ffd-43e6-8677-f23f196ff7c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:57.697666 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d68307-9ffd-43e6-8677-f23f196ff7c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:57.699053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d68307-9ffd-43e6-8677-f23f196ff7c3 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:57.712325 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 274/545] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Aug 23 04:14:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:57.713651 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e23691-7a4a-4ba7-93d6-ea6e048130a2 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:57.892458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5334e-2bad-47e5-ac7f-d3b124652c07 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:57.893153 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5334e-2bad-47e5-ac7f-d3b124652c07 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:57.893640 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5334e-2bad-47e5-ac7f-d3b124652c07 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:57.895415 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 275/546] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:14:57.901325 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48c924-ec39-44d5-92bc-7e06f13d834c None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:57.901959 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48c924-ec39-44d5-92bc-7e06f13d834c None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:57.902525 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48c924-ec39-44d5-92bc-7e06f13d834c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:57.922342 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e23691-7a4a-4ba7-93d6-ea6e048130a2 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:57.923512 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-3c48c924-ec39-44d5-92bc-7e06f13d834c None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:57.925048 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-3c48c924-ec39-44d5-92bc-7e06f13d834c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:57.972300 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:57.991871 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159eeb8b-bd92-408c-bc88-bf06afa600c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:57.992537 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159eeb8b-bd92-408c-bc88-bf06afa600c2 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:57.992816 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159eeb8b-bd92-408c-bc88-bf06afa600c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:57.993167 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159eeb8b-bd92-408c-bc88-bf06afa600c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:58.006231 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159eeb8b-bd92-408c-bc88-bf06afa600c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:58.007782 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159eeb8b-bd92-408c-bc88-bf06afa600c2 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:58.016484 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:58.023699 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-159eeb8b-bd92-408c-bc88-bf06afa600c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:58.023699 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:58.023699 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:58.023699 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:58.023699 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:58.023699 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:58.023699 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:58.023699 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:58.023699 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:58.023699 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:58.023699 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:58.023699 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:58.023699 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:14:58.023699 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:14:58.023699 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:14:58.023699 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:14:58.024646 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:58.024646 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:58.024646 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:58.024646 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:58.024646 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:58.024646 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:58.024646 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:58.024646 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:58.024646 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:58.024646 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:58.024646 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:58.024646 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:58.024646 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:58.024646 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:58.024646 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:58.024646 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:58.024646 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:14:58.025883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:14:58.025883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:58.025883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:58.025883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 272/547] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:14:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:58.031869 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159eeb8b-bd92-408c-bc88-bf06afa600c2 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:58.046987 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 276/548] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 04:14:58.186598 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159eeb8b-bd92-408c-bc88-bf06afa600c2 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:58.233656 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe3479-b911-4d85-b986-2f88377ccaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:58.234266 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe3479-b911-4d85-b986-2f88377ccaef None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:58.234541 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe3479-b911-4d85-b986-2f88377ccaef None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:58.234810 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe3479-b911-4d85-b986-2f88377ccaef None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:58.235449 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe3479-b911-4d85-b986-2f88377ccaef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:58.235748 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe3479-b911-4d85-b986-2f88377ccaef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:58.237147 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe3479-b911-4d85-b986-2f88377ccaef None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:58.257370 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 273/549] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:58.262650 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c48c924-ec39-44d5-92bc-7e06f13d834c None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:58.406662 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c48c924-ec39-44d5-92bc-7e06f13d834c None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:58.421989 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe3479-b911-4d85-b986-2f88377ccaef None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:58.475725 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20baf9d4-0e4a-4828-ba82-42da4263aa70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:58.476390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20baf9d4-0e4a-4828-ba82-42da4263aa70 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:58.476671 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20baf9d4-0e4a-4828-ba82-42da4263aa70 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:58.476932 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20baf9d4-0e4a-4828-ba82-42da4263aa70 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:58.491953 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20baf9d4-0e4a-4828-ba82-42da4263aa70 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:58.493418 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20baf9d4-0e4a-4828-ba82-42da4263aa70 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:58.517295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-20baf9d4-0e4a-4828-ba82-42da4263aa70 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 04:14:58.517295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:58.517295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:58.517295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:58.517295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:58.517295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:58.517295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:58.517295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:58.517295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:58.517295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:58.517295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:58.517295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:58.517295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:14:58.517295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:14:58.517295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:14:58.517295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:14:58.519204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:58.519204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:58.519204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:14:58.519204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:14:58.519204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:14:58.519204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:14:58.519204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:58.519204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:58.519204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:58.519204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:58.519204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:58.519204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:58.519204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:58.519204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:58.519204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:58.519204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:58.519204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:58.519204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:58.520316 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:58.520316 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:58.520316 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:14:58.520316 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:14:58.520316 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:14:58.520316 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:14:58.520316 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:14:58.520316 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:14:58.520316 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 04:14:58.520316 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:58.525082 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 277/550] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:14:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:58.526223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20baf9d4-0e4a-4828-ba82-42da4263aa70 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:58.538707 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:58.569478 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:58.611423 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe3479-b911-4d85-b986-2f88377ccaef None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:58.644087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20baf9d4-0e4a-4828-ba82-42da4263aa70 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:58.684770 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93821e-d2f9-4eab-8440-3e80eca891ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:58.685377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93821e-d2f9-4eab-8440-3e80eca891ea None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:58.685651 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93821e-d2f9-4eab-8440-3e80eca891ea None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:58.685920 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93821e-d2f9-4eab-8440-3e80eca891ea None admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:58.686767 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc93821e-d2f9-4eab-8440-3e80eca891ea None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:58.687069 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc93821e-d2f9-4eab-8440-3e80eca891ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:58.688544 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc93821e-d2f9-4eab-8440-3e80eca891ea None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:58.705524 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c32ee-b8c5-4b9e-97af-34452f23357e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:58.706169 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c32ee-b8c5-4b9e-97af-34452f23357e None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:58.706442 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c32ee-b8c5-4b9e-97af-34452f23357e None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:58.706709 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c32ee-b8c5-4b9e-97af-34452f23357e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:58.709082 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 278/551] 10.222.0.18 () {56 vars in 1135 bytes} [Tue Aug 23 04:14:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:14:58.714639 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93821e-d2f9-4eab-8440-3e80eca891ea None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:58.735048 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c32ee-b8c5-4b9e-97af-34452f23357e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:58.743597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-eb0c32ee-b8c5-4b9e-97af-34452f23357e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:14:58.743597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:58.743597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:58.743597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:58.743597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:58.743597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:58.743597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:58.743597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:58.743597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:58.743597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:58.743597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:58.743597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:58.743597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:58.743597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:58.743597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:58.743597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:58.744539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:58.744539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:58.744539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:58.744539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:58.744539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:58.744539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:58.744539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:58.744539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:58.744539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:58.744539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:58.744539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:58.744539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:58.744539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:58.744539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:58.744539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:58.744539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:58.746076 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:58.746076 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:58.746076 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:58.746076 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:58.746076 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:58.746076 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:58.746076 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:58.746076 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:58.746076 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:58.746076 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:58.746076 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:14:58.746076 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:58.746076 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 274/552] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Aug 23 04:14:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:14:58.751086 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c32ee-b8c5-4b9e-97af-34452f23357e None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:58.851105 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:58.872005 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93821e-d2f9-4eab-8440-3e80eca891ea None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:58.894490 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c32ee-b8c5-4b9e-97af-34452f23357e None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:58.908505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549d94a-6c3a-4a4e-a012-d0fea5305d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:58.909279 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549d94a-6c3a-4a4e-a012-d0fea5305d7a None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:58.909623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549d94a-6c3a-4a4e-a012-d0fea5305d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:58.909965 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549d94a-6c3a-4a4e-a012-d0fea5305d7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:58.921042 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:58.942321 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76962c6-f71a-4cf3-a6d3-f98416590487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:58.943510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76962c6-f71a-4cf3-a6d3-f98416590487 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:58.944635 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76962c6-f71a-4cf3-a6d3-f98416590487 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:58.945249 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76962c6-f71a-4cf3-a6d3-f98416590487 None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:58.946211 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76962c6-f71a-4cf3-a6d3-f98416590487 None admin] RBAC: Adding query filter params (name=service) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:58.946696 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76962c6-f71a-4cf3-a6d3-f98416590487 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:58.949410 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76962c6-f71a-4cf3-a6d3-f98416590487 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:58.961621 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 275/553] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Aug 23 04:14:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:58.963123 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064651f-fbbb-4743-8f14-30bb4c403f42 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:58.963782 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064651f-fbbb-4743-8f14-30bb4c403f42 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:58.964146 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064651f-fbbb-4743-8f14-30bb4c403f42 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:58.970159 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 276/554] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:58.970159 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064651f-fbbb-4743-8f14-30bb4c403f42 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:58.999101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549d94a-6c3a-4a4e-a012-d0fea5305d7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:59.001723 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549d94a-6c3a-4a4e-a012-d0fea5305d7a None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:59.077769 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064651f-fbbb-4743-8f14-30bb4c403f42 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:59.094553 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:59.113395 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f177ebf-b8c8-4cff-aa0c-3955a284516d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:59.114028 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f177ebf-b8c8-4cff-aa0c-3955a284516d None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:59.114310 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f177ebf-b8c8-4cff-aa0c-3955a284516d None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:59.114399 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 279/555] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:14:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:14:59.114584 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f177ebf-b8c8-4cff-aa0c-3955a284516d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:59.115561 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc7e78-c034-4eb6-af8b-c7b38179a6e3 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:59.115843 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc7e78-c034-4eb6-af8b-c7b38179a6e3 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:59.116117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc7e78-c034-4eb6-af8b-c7b38179a6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:59.126105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f177ebf-b8c8-4cff-aa0c-3955a284516d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:59.127557 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f177ebf-b8c8-4cff-aa0c-3955a284516d None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:59.134608 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-93cc7e78-c034-4eb6-af8b-c7b38179a6e3 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:59.135802 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-93cc7e78-c034-4eb6-af8b-c7b38179a6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:59.142220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-9f177ebf-b8c8-4cff-aa0c-3955a284516d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:59.142220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:59.142220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:59.142220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:59.142220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:59.142220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:59.142220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:59.142220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:59.142220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:59.142220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:59.142220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:59.142220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:59.142220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:14:59.142220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:14:59.142220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:14:59.142220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:14:59.143307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:59.143307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:59.143307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:59.143307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:59.143307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:59.143307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:59.143307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:59.143307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:59.143307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:59.143307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:59.143307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:59.143307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:59.143307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:59.143307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:59.143307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:59.143307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:59.143307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:14:59.144411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:14:59.144411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:59.144411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:59.144411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 277/556] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:14:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:59.144823 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc625cac-56bb-427c-8218-b98a6a02eb3a None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:59.145108 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc625cac-56bb-427c-8218-b98a6a02eb3a None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:59.145383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc625cac-56bb-427c-8218-b98a6a02eb3a None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:59.146615 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 278/557] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:59.151012 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711fa76-a5d4-44ee-9a80-e461835bb30f None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:59.151012 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711fa76-a5d4-44ee-9a80-e461835bb30f None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:59.151012 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711fa76-a5d4-44ee-9a80-e461835bb30f None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:59.151012 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 279/558] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:14:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:14:59.151012 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711fa76-a5d4-44ee-9a80-e461835bb30f None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:59.184362 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:59.210217 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 280/559] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:59.211057 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc7e78-c034-4eb6-af8b-c7b38179a6e3 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:59.254877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711fa76-a5d4-44ee-9a80-e461835bb30f None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:59.290158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce6b7d-b9a8-4a6e-9d7c-9455353b027d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:59.290873 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce6b7d-b9a8-4a6e-9d7c-9455353b027d None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:59.291158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce6b7d-b9a8-4a6e-9d7c-9455353b027d None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:59.291432 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce6b7d-b9a8-4a6e-9d7c-9455353b027d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:59.301911 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ce6b7d-b9a8-4a6e-9d7c-9455353b027d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:59.303404 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ce6b7d-b9a8-4a6e-9d7c-9455353b027d None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:59.317327 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-a2ce6b7d-b9a8-4a6e-9d7c-9455353b027d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:59.317327 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:59.317327 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:59.317327 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:59.317327 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:59.317327 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:59.317327 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:59.317327 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:59.317327 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:59.317327 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:59.317327 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:59.317327 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:59.317327 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:14:59.317327 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:14:59.317327 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:14:59.317327 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:14:59.318838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:59.318838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:59.318838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:59.318838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:59.318838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:59.318838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:59.318838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:59.318838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:59.318838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:59.318838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:59.318838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:59.318838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:59.318838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:59.318838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:59.318838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:59.318838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:59.318838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:14:59.319819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:14:59.319819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:14:59.319819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:59.319819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 280/560] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:14:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:59.319819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddacd45-3109-4dab-a612-8696696e8f67 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:59.320158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddacd45-3109-4dab-a612-8696696e8f67 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:59.320227 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddacd45-3109-4dab-a612-8696696e8f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:59.326222 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc7e78-c034-4eb6-af8b-c7b38179a6e3 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:59.338961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-eddacd45-3109-4dab-a612-8696696e8f67 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:59.339151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:59.340195 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-eddacd45-3109-4dab-a612-8696696e8f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:59.364954 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947bcb79-a211-4452-89e2-db24afb021ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:59.366036 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947bcb79-a211-4452-89e2-db24afb021ec None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:59.366340 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947bcb79-a211-4452-89e2-db24afb021ec None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:59.366617 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947bcb79-a211-4452-89e2-db24afb021ec None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:59.367348 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947bcb79-a211-4452-89e2-db24afb021ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:59.367572 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947bcb79-a211-4452-89e2-db24afb021ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:59.368994 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947bcb79-a211-4452-89e2-db24afb021ec None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:59.384812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 281/561] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:59.384812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4186da7-fb8c-4b63-be48-af6db97dedc8 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:59.384812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4186da7-fb8c-4b63-be48-af6db97dedc8 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:59.384812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4186da7-fb8c-4b63-be48-af6db97dedc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:59.400342 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-d4186da7-fb8c-4b63-be48-af6db97dedc8 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:14:59.401512 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-d4186da7-fb8c-4b63-be48-af6db97dedc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:14:59.416858 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 281/562] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:59.418036 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddacd45-3109-4dab-a612-8696696e8f67 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:59.435761 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:59.488143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 282/563] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:14:59.489323 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4186da7-fb8c-4b63-be48-af6db97dedc8 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:59.553957 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddacd45-3109-4dab-a612-8696696e8f67 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:59.591465 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d11acd-db48-45a2-b1cf-21de7286d6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:59.592060 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d11acd-db48-45a2-b1cf-21de7286d6ef None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:59.592430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d11acd-db48-45a2-b1cf-21de7286d6ef None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:59.592791 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d11acd-db48-45a2-b1cf-21de7286d6ef None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:59.593524 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d11acd-db48-45a2-b1cf-21de7286d6ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:59.593940 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d11acd-db48-45a2-b1cf-21de7286d6ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:59.595399 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d11acd-db48-45a2-b1cf-21de7286d6ef None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:59.603516 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:59.608290 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4186da7-fb8c-4b63-be48-af6db97dedc8 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:59.609043 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 282/564] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:59.610006 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d11acd-db48-45a2-b1cf-21de7286d6ef None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:59.649060 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b29c55-24cb-45a3-b904-e40cbb6ce0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:59.649859 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b29c55-24cb-45a3-b904-e40cbb6ce0ff None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:59.650230 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b29c55-24cb-45a3-b904-e40cbb6ce0ff None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:59.650619 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b29c55-24cb-45a3-b904-e40cbb6ce0ff None admin] PATH_INFO: `/v3/users/nova` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:59.658211 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b29c55-24cb-45a3-b904-e40cbb6ce0ff None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:59.659965 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b29c55-24cb-45a3-b904-e40cbb6ce0ff None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:59.672336 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-90b29c55-24cb-45a3-b904-e40cbb6ce0ff None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 04:14:59.672336 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:59.672336 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:59.672336 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:59.672336 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:59.672336 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:59.672336 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:59.672336 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:59.672336 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:59.672336 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:59.672336 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:59.672336 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:59.672336 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:14:59.672336 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:14:59.672336 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:14:59.672336 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:14:59.673560 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:59.673560 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:59.673560 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:14:59.673560 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:14:59.673560 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:14:59.673560 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:14:59.673560 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:59.673560 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:59.673560 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:59.673560 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:59.673560 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:59.673560 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:59.673560 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:59.673560 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:59.673560 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:59.673560 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:59.673560 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:59.673560 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:59.674693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:59.674693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:59.674693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:14:59.674693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:14:59.674693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:14:59.674693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:14:59.674693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:14:59.674693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:14:59.674693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 04:14:59.674693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:59.675722 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 283/565] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Aug 23 04:14:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:59.676677 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b29c55-24cb-45a3-b904-e40cbb6ce0ff None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:59.710119 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:59.734216 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d11acd-db48-45a2-b1cf-21de7286d6ef None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:59.768263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc105f-2709-42bc-8274-4f603654d74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:59.768887 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc105f-2709-42bc-8274-4f603654d74e None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:59.769160 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc105f-2709-42bc-8274-4f603654d74e None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:59.769424 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc105f-2709-42bc-8274-4f603654d74e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:59.779264 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc105f-2709-42bc-8274-4f603654d74e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:59.780840 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b29c55-24cb-45a3-b904-e40cbb6ce0ff None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:59.784352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-3fcc105f-2709-42bc-8274-4f603654d74e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:14:59.784352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:59.784352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:59.784352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:59.784352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:59.784352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:59.784352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:59.784352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:59.784352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:59.784352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:59.784352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:59.784352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:59.784352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:59.784352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:59.784352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:59.784352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:59.785422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:59.785422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:59.785422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:59.785422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:59.785422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:59.785422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:59.785422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:59.785422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:59.785422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:59.785422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:59.785422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:59.785422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:59.785422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:59.785422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:59.785422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:59.785422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:59.786736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:59.786736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:59.786736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:59.786736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:59.786736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:59.786736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:59.786736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:59.786736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:59.786736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:59.786736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:59.786736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:14:59.786736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:14:59.786736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 283/566] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Aug 23 04:14:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:59.786736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc105f-2709-42bc-8274-4f603654d74e None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:59.814541 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d9fbe7-73d9-405a-a20a-8006e3be4a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:59.815098 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d9fbe7-73d9-405a-a20a-8006e3be4a6f None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:59.815370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d9fbe7-73d9-405a-a20a-8006e3be4a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:59.815637 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d9fbe7-73d9-405a-a20a-8006e3be4a6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:59.822332 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d9fbe7-73d9-405a-a20a-8006e3be4a6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:59.823766 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d9fbe7-73d9-405a-a20a-8006e3be4a6f None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:59.834645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-51d9fbe7-73d9-405a-a20a-8006e3be4a6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:14:59.834645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:59.834645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:59.834645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:59.834645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:59.834645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:59.834645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:59.834645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:59.834645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:59.834645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:59.834645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:59.834645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:59.834645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:14:59.834645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:14:59.834645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:14:59.835669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:14:59.835669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:59.835669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:59.835669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:59.835669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:59.835669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:59.835669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:59.835669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:59.835669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:59.835669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:59.835669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:59.835669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:59.835669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:59.835669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:59.835669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:59.835669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:59.835669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:59.836589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:14:59.836589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:14:59.836589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:14:59.836589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:14:59.836589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:14:59.836589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:14:59.836589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:14:59.836589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:59.836589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 284/567] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:14:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:59.837505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d9fbe7-73d9-405a-a20a-8006e3be4a6f None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:59.861405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:59.892409 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc105f-2709-42bc-8274-4f603654d74e None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:59.927733 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d7bdfe-c752-4ab8-9a81-c2f0d0df7a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:59.928358 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d7bdfe-c752-4ab8-9a81-c2f0d0df7a04 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:59.928639 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d7bdfe-c752-4ab8-9a81-c2f0d0df7a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:59.928926 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d7bdfe-c752-4ab8-9a81-c2f0d0df7a04 None admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:59.929660 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d7bdfe-c752-4ab8-9a81-c2f0d0df7a04 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:14:59.929960 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d7bdfe-c752-4ab8-9a81-c2f0d0df7a04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:14:59.931395 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d7bdfe-c752-4ab8-9a81-c2f0d0df7a04 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:14:59.934462 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:14:59.941902 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d9fbe7-73d9-405a-a20a-8006e3be4a6f None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:14:59.949054 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 284/568] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Aug 23 04:14:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:14:59.949863 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d7bdfe-c752-4ab8-9a81-c2f0d0df7a04 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:14:59.977696 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7ac4d-fc3f-4302-bea6-1a2e4cad3f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:14:59.978712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7ac4d-fc3f-4302-bea6-1a2e4cad3f54 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:14:59.979200 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7ac4d-fc3f-4302-bea6-1a2e4cad3f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:14:59.979683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7ac4d-fc3f-4302-bea6-1a2e4cad3f54 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:14:59.991879 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c7ac4d-fc3f-4302-bea6-1a2e4cad3f54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:14:59.997028 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-01c7ac4d-fc3f-4302-bea6-1a2e4cad3f54 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:14:59.997028 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:14:59.997028 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:14:59.997028 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:14:59.997028 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:14:59.997028 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:14:59.997028 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:14:59.997028 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:14:59.997028 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:14:59.997028 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:14:59.997028 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:14:59.997028 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:14:59.997028 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:14:59.997028 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:14:59.997028 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:14:59.997028 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:14:59.997981 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:14:59.997981 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:14:59.997981 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:14:59.997981 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:14:59.997981 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:14:59.997981 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:14:59.997981 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:14:59.997981 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:14:59.997981 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:14:59.997981 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:14:59.997981 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:14:59.997981 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:14:59.997981 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:14:59.997981 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:14:59.997981 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:14:59.997981 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:14:59.998801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:14:59.998801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:14:59.998801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:14:59.998801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:14:59.998801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:14:59.998801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:14:59.998801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:14:59.998801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:14:59.998801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:14:59.998801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:14:59.998801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:14:59.998801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:14:59.998801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 285/569] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Aug 23 04:14:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:14:59.999496 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7ac4d-fc3f-4302-bea6-1a2e4cad3f54 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:00.056450 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d7bdfe-c752-4ab8-9a81-c2f0d0df7a04 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:00.084165 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:00.091364 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8014c3e3-074c-4c13-aa10-34dceebe2059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:00.092023 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014c3e3-074c-4c13-aa10-34dceebe2059 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:00.092298 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014c3e3-074c-4c13-aa10-34dceebe2059 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:00.092564 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014c3e3-074c-4c13-aa10-34dceebe2059 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:00.102501 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8014c3e3-074c-4c13-aa10-34dceebe2059 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:15:00.103636 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7ac4d-fc3f-4302-bea6-1a2e4cad3f54 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:00.107082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-8014c3e3-074c-4c13-aa10-34dceebe2059 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:15:00.107082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:00.107082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:00.107082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:00.107082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:00.107082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:00.107082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:00.107082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:00.107082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:00.107082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:00.107082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:00.107082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:00.107082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:15:00.107082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:15:00.107082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:15:00.107082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:15:00.108000 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:15:00.108000 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:15:00.108000 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:15:00.108000 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:15:00.108000 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:00.108000 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:00.108000 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:00.108000 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:00.108000 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:00.108000 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:00.108000 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:00.108000 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:00.108000 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:00.108000 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:00.108000 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:00.108000 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:00.108798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:00.108798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:00.108798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:00.108798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:00.108798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:15:00.108798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:15:00.108798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:15:00.108798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:15:00.108798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:15:00.108798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:15:00.108798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:15:00.108798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:00.108798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 285/570] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Aug 23 04:14:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:00.109455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8014c3e3-074c-4c13-aa10-34dceebe2059 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:00.141627 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bdefc6-5741-4c85-bd15-03d2b638620a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:00.142230 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdefc6-5741-4c85-bd15-03d2b638620a None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:00.142503 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdefc6-5741-4c85-bd15-03d2b638620a None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:00.142772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdefc6-5741-4c85-bd15-03d2b638620a None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:00.143416 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bdefc6-5741-4c85-bd15-03d2b638620a None admin] RBAC: Adding query filter params (name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:00.143714 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bdefc6-5741-4c85-bd15-03d2b638620a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:00.145127 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bdefc6-5741-4c85-bd15-03d2b638620a None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:00.153229 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 286/571] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Aug 23 04:14:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:00.154035 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bdefc6-5741-4c85-bd15-03d2b638620a None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:00.173683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:00.219200 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8014c3e3-074c-4c13-aa10-34dceebe2059 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:00.252637 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c86c89-1e99-4333-8be5-dd8c0ff492ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:00.253266 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c86c89-1e99-4333-8be5-dd8c0ff492ee None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:00.253539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c86c89-1e99-4333-8be5-dd8c0ff492ee None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:00.253803 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c86c89-1e99-4333-8be5-dd8c0ff492ee None admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:00.254382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bdefc6-5741-4c85-bd15-03d2b638620a None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:00.254480 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c86c89-1e99-4333-8be5-dd8c0ff492ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:00.254764 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c86c89-1e99-4333-8be5-dd8c0ff492ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:00.256487 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c86c89-1e99-4333-8be5-dd8c0ff492ee None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:00.268251 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 286/572] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:15:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:00.269042 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c86c89-1e99-4333-8be5-dd8c0ff492ee None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:00.288634 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5ba86-c331-40ec-89d6-a651fcad6c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:00.289260 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5ba86-c331-40ec-89d6-a651fcad6c38 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:00.289532 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5ba86-c331-40ec-89d6-a651fcad6c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:00.289802 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5ba86-c331-40ec-89d6-a651fcad6c38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:00.299962 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a5ba86-c331-40ec-89d6-a651fcad6c38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:00.301429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a5ba86-c331-40ec-89d6-a651fcad6c38 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:00.303138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:00.316374 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-95a5ba86-c331-40ec-89d6-a651fcad6c38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:00.316374 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:00.316374 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:00.316374 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:00.316374 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:00.316374 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:00.316374 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:00.316374 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:00.316374 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:00.316374 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:00.316374 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:00.316374 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:00.316374 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:00.316374 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:00.316374 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:00.316374 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:00.317333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:00.317333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:00.317333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:00.317333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:00.317333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:00.317333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:00.317333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:00.317333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:00.317333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:00.317333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:00.317333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:00.317333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:00.317333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:00.317333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:00.317333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:00.317333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:00.317333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:00.318223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:00.318223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:00.318223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:00.318223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 287/573] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:00.318223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5ba86-c331-40ec-89d6-a651fcad6c38 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:00.368787 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c86c89-1e99-4333-8be5-dd8c0ff492ee None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:00.382626 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:00.408253 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a76edc-0209-4dc0-953a-664c89d528c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:00.408862 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a76edc-0209-4dc0-953a-664c89d528c3 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:00.409126 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a76edc-0209-4dc0-953a-664c89d528c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:00.409397 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a76edc-0209-4dc0-953a-664c89d528c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:00.410059 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a76edc-0209-4dc0-953a-664c89d528c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:00.410359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a76edc-0209-4dc0-953a-664c89d528c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:00.411751 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a76edc-0209-4dc0-953a-664c89d528c3 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:00.419727 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 287/574] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Aug 23 04:15:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:00.420617 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a76edc-0209-4dc0-953a-664c89d528c3 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:00.420752 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5ba86-c331-40ec-89d6-a651fcad6c38 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:00.454987 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de320244-d904-4345-9395-ef4e3a04241a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:00.455603 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de320244-d904-4345-9395-ef4e3a04241a None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:00.455885 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de320244-d904-4345-9395-ef4e3a04241a None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:00.456189 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de320244-d904-4345-9395-ef4e3a04241a None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:00.457001 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de320244-d904-4345-9395-ef4e3a04241a None admin] RBAC: Adding query filter params (name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:00.457221 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de320244-d904-4345-9395-ef4e3a04241a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:00.459986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de320244-d904-4345-9395-ef4e3a04241a None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:00.468943 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 288/575] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Aug 23 04:15:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:00.469752 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de320244-d904-4345-9395-ef4e3a04241a None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:00.524514 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:00.533327 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a76edc-0209-4dc0-953a-664c89d528c3 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:00.566200 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a683335-b0d7-49e8-93e4-5538c6f7bdeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:00.566834 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a683335-b0d7-49e8-93e4-5538c6f7bdeb None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:00.567109 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a683335-b0d7-49e8-93e4-5538c6f7bdeb None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:00.567376 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a683335-b0d7-49e8-93e4-5538c6f7bdeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:00.568173 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de320244-d904-4345-9395-ef4e3a04241a None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:00.577698 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a683335-b0d7-49e8-93e4-5538c6f7bdeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:00.579141 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a683335-b0d7-49e8-93e4-5538c6f7bdeb None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:00.593106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-0a683335-b0d7-49e8-93e4-5538c6f7bdeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:00.593106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:00.593106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:00.593106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:00.593106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:00.593106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:00.593106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:00.593106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:00.593106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:00.593106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:00.593106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:00.593106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:00.593106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:00.593106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:00.593106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:00.593106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:00.594184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:00.594184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:00.594184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:00.594184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:00.594184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:00.594184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:00.594184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:00.594184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:00.594184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:00.594184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:00.594184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:00.594184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:00.594184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:00.594184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:00.594184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:00.594184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:00.594184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:00.595498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:00.595498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:00.595498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:00.595498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 288/576] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:00.595498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a683335-b0d7-49e8-93e4-5538c6f7bdeb None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:00.603674 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d71537-5642-4eb7-b5a6-65f7eb417344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:00.604390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d71537-5642-4eb7-b5a6-65f7eb417344 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:00.604673 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d71537-5642-4eb7-b5a6-65f7eb417344 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:00.604946 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d71537-5642-4eb7-b5a6-65f7eb417344 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:00.605677 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d71537-5642-4eb7-b5a6-65f7eb417344 None admin] RBAC: Adding query filter params (role.id=bdf71ef6ab10430e8c356cd4d952ff5c, scope.project.id=9f4373dfb5f9425eb73669e24d18a08f, user.id=3f1418fd211a448da1a50175dce13b32) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:00.605981 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d71537-5642-4eb7-b5a6-65f7eb417344 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:00.607409 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d71537-5642-4eb7-b5a6-65f7eb417344 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:00.620517 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 289/577] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:15:00 2022] GET /identity/v3/role_assignments?user.id=3f1418fd211a448da1a50175dce13b32&scope.project.id=9f4373dfb5f9425eb73669e24d18a08f&role.id=bdf71ef6ab10430e8c356cd4d952ff5c => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:00.620517 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d71537-5642-4eb7-b5a6-65f7eb417344 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:00.624219 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:00.707994 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a683335-b0d7-49e8-93e4-5538c6f7bdeb None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:00.735082 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d71537-5642-4eb7-b5a6-65f7eb417344 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:00.746792 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497caf82-e350-41a2-841a-9697660b9c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:00.747418 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497caf82-e350-41a2-841a-9697660b9c4c None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:00.747688 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497caf82-e350-41a2-841a-9697660b9c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:00.747981 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497caf82-e350-41a2-841a-9697660b9c4c None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:00.748662 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497caf82-e350-41a2-841a-9697660b9c4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:00.748961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497caf82-e350-41a2-841a-9697660b9c4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:00.750379 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497caf82-e350-41a2-841a-9697660b9c4c None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:00.763140 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 289/578] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:00.764012 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497caf82-e350-41a2-841a-9697660b9c4c None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:00.764859 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:00.783017 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2a44b-bab0-42f8-bf44-4c54498fcd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:00.783856 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2a44b-bab0-42f8-bf44-4c54498fcd27 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:00.784070 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2a44b-bab0-42f8-bf44-4c54498fcd27 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:00.784245 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2a44b-bab0-42f8-bf44-4c54498fcd27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:00.795276 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2a44b-bab0-42f8-bf44-4c54498fcd27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:00.796951 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2a44b-bab0-42f8-bf44-4c54498fcd27 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:00.811390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-63e2a44b-bab0-42f8-bf44-4c54498fcd27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:00.811390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:00.811390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:00.811390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:00.811390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:00.811390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:00.811390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:00.811390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:00.811390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:00.811390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:00.811390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:00.811390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:00.811390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:00.811390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:00.811390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:00.811390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:00.812445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:00.812445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:00.812445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:00.812445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:00.812445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:00.812445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:00.812445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:00.812445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:00.812445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:00.812445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:00.812445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:00.812445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:00.812445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:00.812445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:00.812445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:00.812445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:00.812445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:00.813295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:00.813295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:00.813295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:00.813295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 290/579] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:00.813295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2a44b-bab0-42f8-bf44-4c54498fcd27 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:00.863590 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:00.877592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497caf82-e350-41a2-841a-9697660b9c4c None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:00.914856 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c827de0-581b-495f-a931-dae3a61081e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:00.915484 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c827de0-581b-495f-a931-dae3a61081e9 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:00.915758 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c827de0-581b-495f-a931-dae3a61081e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:00.916056 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c827de0-581b-495f-a931-dae3a61081e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:00.916344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2a44b-bab0-42f8-bf44-4c54498fcd27 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:00.927663 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c827de0-581b-495f-a931-dae3a61081e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:00.929315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c827de0-581b-495f-a931-dae3a61081e9 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:00.944828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-6c827de0-581b-495f-a931-dae3a61081e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:00.944828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:00.944828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:00.944828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:00.944828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:00.944828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:00.944828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:00.944828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:00.944828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:00.944828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:00.944828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:00.944828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:00.944828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:00.944828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:00.944828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:00.944828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:00.945787 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:00.945787 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:00.945787 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:00.945787 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:00.945787 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:00.945787 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:00.945787 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:00.945787 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:00.945787 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:00.945787 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:00.945787 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:00.945787 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:00.945787 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:00.945787 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:00.945787 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:00.945787 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:00.945787 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:00.946702 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:00.946702 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:00.946702 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:00.946702 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 290/580] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:00.946916 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c827de0-581b-495f-a931-dae3a61081e9 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:00.954900 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce4bfd-e1f7-4ad8-9419-8f9189023875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:00.955514 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce4bfd-e1f7-4ad8-9419-8f9189023875 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:00.955790 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce4bfd-e1f7-4ad8-9419-8f9189023875 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:00.956082 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce4bfd-e1f7-4ad8-9419-8f9189023875 None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:00.956726 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ce4bfd-e1f7-4ad8-9419-8f9189023875 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:00.957016 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ce4bfd-e1f7-4ad8-9419-8f9189023875 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:00.958458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ce4bfd-e1f7-4ad8-9419-8f9189023875 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:00.976427 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 291/581] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:00.976427 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce4bfd-e1f7-4ad8-9419-8f9189023875 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:01.007053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:01.065844 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c827de0-581b-495f-a931-dae3a61081e9 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:01.078327 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce4bfd-e1f7-4ad8-9419-8f9189023875 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:01.104781 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade61b98-eff7-4db4-996b-3fad97a38386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:01.105194 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade61b98-eff7-4db4-996b-3fad97a38386 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:01.105458 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade61b98-eff7-4db4-996b-3fad97a38386 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:01.105820 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade61b98-eff7-4db4-996b-3fad97a38386 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:01.112619 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade61b98-eff7-4db4-996b-3fad97a38386 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:01.114070 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade61b98-eff7-4db4-996b-3fad97a38386 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:01.115964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7fc7f-d54e-45c6-9026-42ab5c29911b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:01.116629 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:01.116957 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7fc7f-d54e-45c6-9026-42ab5c29911b None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:01.117238 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7fc7f-d54e-45c6-9026-42ab5c29911b None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:01.117510 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7fc7f-d54e-45c6-9026-42ab5c29911b None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:01.118147 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7fc7f-d54e-45c6-9026-42ab5c29911b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:01.118504 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7fc7f-d54e-45c6-9026-42ab5c29911b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:01.119800 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7fc7f-d54e-45c6-9026-42ab5c29911b None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:01.126587 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-ade61b98-eff7-4db4-996b-3fad97a38386 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:01.126587 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:01.126587 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:01.126587 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:01.126587 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:01.126587 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:01.126587 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:01.126587 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:01.126587 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:01.126587 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:01.126587 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:01.126587 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:01.126587 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:15:01.126587 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:15:01.126587 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:15:01.127494 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:15:01.127494 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:01.127494 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:01.127494 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:01.127494 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:01.127494 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:01.127494 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:01.127494 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:01.127494 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:01.127494 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:01.127494 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:01.127494 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:01.127494 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:01.127494 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:01.127494 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:01.127494 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:01.127494 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:01.128477 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:15:01.128477 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:15:01.128477 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:15:01.128477 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:15:01.128477 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:15:01.128477 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:15:01.128477 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:01.128477 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:01.136214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 291/582] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:15:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:01.136214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade61b98-eff7-4db4-996b-3fad97a38386 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:01.143838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 292/583] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:01.143838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7fc7f-d54e-45c6-9026-42ab5c29911b None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:01.238369 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:01.238697 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade61b98-eff7-4db4-996b-3fad97a38386 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:01.249066 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7fc7f-d54e-45c6-9026-42ab5c29911b None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:01.274733 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cdd9bd-80a8-4663-adac-76e75f0c16b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:01.275348 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cdd9bd-80a8-4663-adac-76e75f0c16b1 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:01.275633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cdd9bd-80a8-4663-adac-76e75f0c16b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:01.275895 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cdd9bd-80a8-4663-adac-76e75f0c16b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:01.276576 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cdd9bd-80a8-4663-adac-76e75f0c16b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:01.276878 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cdd9bd-80a8-4663-adac-76e75f0c16b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:01.278243 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cdd9bd-80a8-4663-adac-76e75f0c16b1 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:01.283387 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2623cfd0-e991-4c42-8d51-1ce743f18617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:01.284044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623cfd0-e991-4c42-8d51-1ce743f18617 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:01.284322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623cfd0-e991-4c42-8d51-1ce743f18617 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:01.284593 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623cfd0-e991-4c42-8d51-1ce743f18617 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:01.292899 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2623cfd0-e991-4c42-8d51-1ce743f18617 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:01.298800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 292/584] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:01.298965 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2623cfd0-e991-4c42-8d51-1ce743f18617 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:01.299059 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cdd9bd-80a8-4663-adac-76e75f0c16b1 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:01.313264 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-2623cfd0-e991-4c42-8d51-1ce743f18617 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 23 04:15:01.313264 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:01.313264 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:01.313264 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:01.313264 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:01.313264 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:01.313264 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:01.313264 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:01.313264 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:01.313264 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:01.313264 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:01.313264 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:01.313264 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:15:01.313264 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:15:01.313264 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:15:01.313264 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:15:01.314248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:01.314248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:01.314248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:15:01.314248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:01.314248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:15:01.314248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:01.314248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:01.314248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:01.314248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:01.314248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:01.314248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:01.314248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:01.314248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:01.314248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:01.314248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:01.314248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:01.314248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:01.314248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:01.315161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:01.315161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:01.315161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:15:01.315161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:15:01.315161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:15:01.315161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:15:01.315161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:15:01.315161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:15:01.315161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 23 04:15:01.315161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:01.315161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 293/585] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:15:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:01.315161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2623cfd0-e991-4c42-8d51-1ce743f18617 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:01.363832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:01.418435 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cdd9bd-80a8-4663-adac-76e75f0c16b1 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:01.425081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2623cfd0-e991-4c42-8d51-1ce743f18617 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:01.456103 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa80087-d971-4d67-b0f4-ed2e22d60010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:01.456727 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa80087-d971-4d67-b0f4-ed2e22d60010 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:01.457007 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa80087-d971-4d67-b0f4-ed2e22d60010 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:01.457630 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa80087-d971-4d67-b0f4-ed2e22d60010 None admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:01.457969 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa80087-d971-4d67-b0f4-ed2e22d60010 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:01.458274 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa80087-d971-4d67-b0f4-ed2e22d60010 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:01.460289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa80087-d971-4d67-b0f4-ed2e22d60010 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:01.469576 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ed042-1fa8-4985-ab0e-62ffb7f27ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:01.471037 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ed042-1fa8-4985-ab0e-62ffb7f27ba3 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:01.471545 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ed042-1fa8-4985-ab0e-62ffb7f27ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:01.472252 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ed042-1fa8-4985-ab0e-62ffb7f27ba3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:01.473836 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 293/586] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:15:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:01.474648 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa80087-d971-4d67-b0f4-ed2e22d60010 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:01.486283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ed042-1fa8-4985-ab0e-62ffb7f27ba3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:01.487776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ed042-1fa8-4985-ab0e-62ffb7f27ba3 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:01.500047 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-ff1ed042-1fa8-4985-ab0e-62ffb7f27ba3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 04:15:01.500047 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:01.500047 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:01.500047 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:01.500047 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:01.500047 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:01.500047 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:01.500047 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:01.500047 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:01.500047 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:01.500047 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:01.500047 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:01.500047 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:15:01.500047 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:15:01.500047 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:15:01.500047 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:15:01.501156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:01.501156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:01.501156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:15:01.501156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:01.501156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:15:01.501156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:01.501156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:01.501156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:01.501156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:01.501156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:01.501156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:01.501156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:01.501156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:01.501156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:01.501156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:01.501156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:01.501156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:01.501156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:01.502276 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:01.502276 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:01.502276 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:15:01.502276 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:15:01.502276 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:15:01.502276 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:15:01.502276 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:15:01.502276 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:15:01.502276 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 04:15:01.502276 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:01.502276 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 294/587] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Aug 23 04:15:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:01.502276 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ed042-1fa8-4985-ab0e-62ffb7f27ba3 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:01.518325 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:01.586041 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa80087-d971-4d67-b0f4-ed2e22d60010 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:01.606725 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ed042-1fa8-4985-ab0e-62ffb7f27ba3 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:01.607601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:01.622653 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f00815-4384-4a00-a2f1-dac2b4b40fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:01.623367 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f00815-4384-4a00-a2f1-dac2b4b40fbc None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:01.623716 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f00815-4384-4a00-a2f1-dac2b4b40fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:01.623930 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f00815-4384-4a00-a2f1-dac2b4b40fbc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:01.632659 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f00815-4384-4a00-a2f1-dac2b4b40fbc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:01.634119 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f00815-4384-4a00-a2f1-dac2b4b40fbc None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:01.643981 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676240b-604f-41b7-b4e3-02debb9783b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:01.644599 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676240b-604f-41b7-b4e3-02debb9783b3 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:01.644879 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676240b-604f-41b7-b4e3-02debb9783b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:01.645155 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676240b-604f-41b7-b4e3-02debb9783b3 None admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:01.645250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-04f00815-4384-4a00-a2f1-dac2b4b40fbc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 04:15:01.645250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:01.645250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:01.645250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:01.645250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:01.645250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:01.645250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:01.645250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:01.645250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:01.645250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:01.645250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:01.645250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:01.645250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:15:01.645250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:15:01.645250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:15:01.645250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:15:01.646155 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:01.646155 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:01.646155 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:15:01.646155 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:01.646155 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:15:01.646155 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:01.646155 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:01.646155 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:01.646155 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:01.646155 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:01.646155 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:01.646155 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:01.646155 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:01.646155 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:01.646155 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:01.646155 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:01.646155 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:01.646155 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:01.652362 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:01.652362 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:01.652362 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:15:01.652362 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:15:01.652362 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:15:01.652362 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:15:01.652362 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:15:01.652362 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:15:01.652362 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 04:15:01.652362 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:01.652885 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a676240b-604f-41b7-b4e3-02debb9783b3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:01.652885 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a676240b-604f-41b7-b4e3-02debb9783b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:01.653034 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 294/588] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:15:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:01.653034 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f00815-4384-4a00-a2f1-dac2b4b40fbc None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:01.653152 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a676240b-604f-41b7-b4e3-02debb9783b3 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:01.674591 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 295/589] 10.222.0.18 () {56 vars in 1137 bytes} [Tue Aug 23 04:15:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:01.675934 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676240b-604f-41b7-b4e3-02debb9783b3 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:01.753032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:01.763869 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f00815-4384-4a00-a2f1-dac2b4b40fbc None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:01.786443 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676240b-604f-41b7-b4e3-02debb9783b3 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:01.798662 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aad8c7-c790-465a-af9c-3f3f3eebb014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:01.799390 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aad8c7-c790-465a-af9c-3f3f3eebb014 None admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:01.799666 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aad8c7-c790-465a-af9c-3f3f3eebb014 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:01.799955 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aad8c7-c790-465a-af9c-3f3f3eebb014 None admin] PATH_INFO: `/v3/projects/9f4373dfb5f9425eb73669e24d18a08f/users/cd7b11f479a54adc8d9b8641859b07a1/roles/bdf71ef6ab10430e8c356cd4d952ff5c` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:01.814349 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aad8c7-c790-465a-af9c-3f3f3eebb014 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f4373dfb5f9425eb73669e24d18a08f, user_id=cd7b11f479a54adc8d9b8641859b07a1, role_id=bdf71ef6ab10430e8c356cd4d952ff5c)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:01.815789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aad8c7-c790-465a-af9c-3f3f3eebb014 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:01.824254 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c0986-51ec-4d2d-a28b-5023c9a4ce50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:01.824862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c0986-51ec-4d2d-a28b-5023c9a4ce50 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:01.825140 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c0986-51ec-4d2d-a28b-5023c9a4ce50 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:01.825453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c0986-51ec-4d2d-a28b-5023c9a4ce50 None admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:01.826415 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0c0986-51ec-4d2d-a28b-5023c9a4ce50 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:01.826722 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0c0986-51ec-4d2d-a28b-5023c9a4ce50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:01.828258 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0c0986-51ec-4d2d-a28b-5023c9a4ce50 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:01.831144 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 295/590] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Aug 23 04:15:01 2022] PUT /identity/v3/projects/9f4373dfb5f9425eb73669e24d18a08f/users/cd7b11f479a54adc8d9b8641859b07a1/roles/bdf71ef6ab10430e8c356cd4d952ff5c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:15:01.833005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aad8c7-c790-465a-af9c-3f3f3eebb014 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:01.854081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 296/591] 10.222.0.18 () {56 vars in 1139 bytes} [Tue Aug 23 04:15:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:01.855245 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c0986-51ec-4d2d-a28b-5023c9a4ce50 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:01.877081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:01.966199 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aad8c7-c790-465a-af9c-3f3f3eebb014 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:01.983644 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c0986-51ec-4d2d-a28b-5023c9a4ce50 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:02.006121 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ccbafc-2240-4764-8309-162f7e0bd632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:02.007020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccbafc-2240-4764-8309-162f7e0bd632 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:02.007889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccbafc-2240-4764-8309-162f7e0bd632 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:02.008203 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccbafc-2240-4764-8309-162f7e0bd632 None admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:02.009025 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ccbafc-2240-4764-8309-162f7e0bd632 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:02.009303 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ccbafc-2240-4764-8309-162f7e0bd632 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:02.011191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ccbafc-2240-4764-8309-162f7e0bd632 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:02.014834 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:02.024535 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e07340-0c70-44a6-ac3d-52181837ebad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:02.025417 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e07340-0c70-44a6-ac3d-52181837ebad None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:02.025836 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e07340-0c70-44a6-ac3d-52181837ebad None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:02.026250 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e07340-0c70-44a6-ac3d-52181837ebad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:02.037876 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 296/592] 10.222.0.18 () {56 vars in 1137 bytes} [Tue Aug 23 04:15:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:02.037876 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ccbafc-2240-4764-8309-162f7e0bd632 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:02.045752 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e07340-0c70-44a6-ac3d-52181837ebad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:02.047870 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e07340-0c70-44a6-ac3d-52181837ebad None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:02.067405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-58e07340-0c70-44a6-ac3d-52181837ebad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:02.067405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:02.067405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:02.067405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:02.067405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:02.067405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:02.067405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:02.067405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:02.067405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:02.067405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:02.067405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:02.067405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:02.067405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:02.067405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:02.067405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:02.067405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:02.068686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:02.068686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:02.068686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:02.068686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:02.068686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:02.068686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:02.068686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:02.068686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:02.068686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:02.068686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:02.068686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:02.068686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:02.068686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:02.068686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:02.068686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:02.068686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:02.068686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:02.069698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:02.069698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:02.069698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:02.071990 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 297/593] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:02.072990 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e07340-0c70-44a6-ac3d-52181837ebad None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:02.141402 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:02.155374 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ccbafc-2240-4764-8309-162f7e0bd632 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:02.192217 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d63718-dcf2-4034-be5d-8aedcda095ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:02.192947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d63718-dcf2-4034-be5d-8aedcda095ea None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:02.193288 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d63718-dcf2-4034-be5d-8aedcda095ea None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:02.194080 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d63718-dcf2-4034-be5d-8aedcda095ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:02.204657 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d63718-dcf2-4034-be5d-8aedcda095ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:02.206238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d63718-dcf2-4034-be5d-8aedcda095ea None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:02.209548 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e07340-0c70-44a6-ac3d-52181837ebad None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:02.223800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-67d63718-dcf2-4034-be5d-8aedcda095ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:02.223800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:02.223800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:02.223800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:02.223800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:02.223800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:02.223800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:02.223800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:02.223800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:02.223800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:02.223800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:02.223800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:02.223800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:02.223800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:02.223800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:02.223800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:02.224932 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:02.224932 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:02.224932 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:02.224932 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:02.224932 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:02.224932 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:02.224932 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:02.224932 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:02.224932 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:02.224932 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:02.224932 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:02.224932 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:02.224932 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:02.224932 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:02.224932 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:02.224932 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:02.224932 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:02.225956 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:02.225956 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:02.225956 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:02.229993 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 297/594] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:02.230997 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d63718-dcf2-4034-be5d-8aedcda095ea None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:02.255367 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be959b7b-83b5-4da4-87f5-69e6249d8ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:02.256614 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be959b7b-83b5-4da4-87f5-69e6249d8ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:02.257103 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be959b7b-83b5-4da4-87f5-69e6249d8ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:02.257733 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be959b7b-83b5-4da4-87f5-69e6249d8ce6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:02.272587 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be959b7b-83b5-4da4-87f5-69e6249d8ce6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:02.274844 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be959b7b-83b5-4da4-87f5-69e6249d8ce6 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:02.294995 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-be959b7b-83b5-4da4-87f5-69e6249d8ce6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:02.294995 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:02.294995 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:02.294995 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:02.294995 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:02.294995 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:02.294995 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:02.294995 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:02.294995 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:02.294995 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:02.294995 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:02.294995 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:02.294995 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:02.294995 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:02.294995 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:02.294995 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:02.296031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:02.296031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:02.296031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:02.296031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:02.296031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:02.296031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:02.296031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:02.296031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:02.296031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:02.296031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:02.296031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:02.296031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:02.296031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:02.296031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:02.296031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:02.296031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:02.296031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:02.296913 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:02.296913 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:02.296913 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:02.301299 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 298/595] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:02.302217 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be959b7b-83b5-4da4-87f5-69e6249d8ce6 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:02.336984 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:02.353377 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d63718-dcf2-4034-be5d-8aedcda095ea None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:02.397841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92818381-ba3e-4e3a-af54-902ba2897172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:02.399185 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92818381-ba3e-4e3a-af54-902ba2897172 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:02.399745 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92818381-ba3e-4e3a-af54-902ba2897172 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:02.400223 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92818381-ba3e-4e3a-af54-902ba2897172 None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:02.401198 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92818381-ba3e-4e3a-af54-902ba2897172 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:02.401702 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92818381-ba3e-4e3a-af54-902ba2897172 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:02.403882 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92818381-ba3e-4e3a-af54-902ba2897172 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:02.416863 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:02.421922 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 298/596] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:02.423115 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92818381-ba3e-4e3a-af54-902ba2897172 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:02.423618 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be959b7b-83b5-4da4-87f5-69e6249d8ce6 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:02.475182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5676a3c-ba10-4984-9055-320fa8ef3ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:02.477566 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5676a3c-ba10-4984-9055-320fa8ef3ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:02.477927 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5676a3c-ba10-4984-9055-320fa8ef3ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:02.478282 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5676a3c-ba10-4984-9055-320fa8ef3ce3 None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:02.479023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5676a3c-ba10-4984-9055-320fa8ef3ce3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:02.479396 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5676a3c-ba10-4984-9055-320fa8ef3ce3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:02.482952 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5676a3c-ba10-4984-9055-320fa8ef3ce3 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:02.502072 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 299/597] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:02.503186 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5676a3c-ba10-4984-9055-320fa8ef3ce3 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:02.549894 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92818381-ba3e-4e3a-af54-902ba2897172 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:02.587197 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d725dc9-0c97-4a30-8fc1-d24729b75cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:02.587824 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d725dc9-0c97-4a30-8fc1-d24729b75cef None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:02.588110 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d725dc9-0c97-4a30-8fc1-d24729b75cef None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:02.588749 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d725dc9-0c97-4a30-8fc1-d24729b75cef None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:02.589400 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d725dc9-0c97-4a30-8fc1-d24729b75cef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:02.589704 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d725dc9-0c97-4a30-8fc1-d24729b75cef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:02.591092 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d725dc9-0c97-4a30-8fc1-d24729b75cef None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:02.607527 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 299/598] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:02.607527 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d725dc9-0c97-4a30-8fc1-d24729b75cef None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:02.612639 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:02.623434 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5676a3c-ba10-4984-9055-320fa8ef3ce3 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:02.658963 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36019d-51f4-43be-b6d5-c88631960ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:02.659851 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36019d-51f4-43be-b6d5-c88631960ebd None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:02.660372 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36019d-51f4-43be-b6d5-c88631960ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:02.660838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36019d-51f4-43be-b6d5-c88631960ebd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:02.667461 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:02.667947 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf36019d-51f4-43be-b6d5-c88631960ebd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:02.669443 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf36019d-51f4-43be-b6d5-c88631960ebd None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:02.680887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-cf36019d-51f4-43be-b6d5-c88631960ebd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:02.680887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:02.680887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:02.680887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:02.680887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:02.680887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:02.680887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:02.680887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:02.680887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:02.680887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:02.680887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:02.680887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:02.680887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:15:02.680887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:15:02.680887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:15:02.681799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:15:02.681799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:02.681799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:02.681799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:02.681799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:02.681799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:02.681799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:02.681799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:02.681799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:02.681799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:02.681799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:02.681799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:02.681799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:02.681799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:02.681799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:02.681799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:02.681799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:02.682659 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:15:02.682659 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:15:02.682659 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:15:02.682659 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:15:02.682659 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:15:02.682659 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:15:02.682659 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:02.682659 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:02.686773 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 300/599] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:15:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:02.687777 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12688d-09f6-4184-914f-5a543cb933c9 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:02.688088 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12688d-09f6-4184-914f-5a543cb933c9 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:02.688360 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12688d-09f6-4184-914f-5a543cb933c9 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:02.689848 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 301/600] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:02.690415 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12688d-09f6-4184-914f-5a543cb933c9 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:02.715875 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d725dc9-0c97-4a30-8fc1-d24729b75cef None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:02.754031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1e820-98c8-4459-85b3-43fad0359bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:02.754584 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1e820-98c8-4459-85b3-43fad0359bcb None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:02.754874 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1e820-98c8-4459-85b3-43fad0359bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:02.755132 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1e820-98c8-4459-85b3-43fad0359bcb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:02.761944 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc1e820-98c8-4459-85b3-43fad0359bcb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:02.763381 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc1e820-98c8-4459-85b3-43fad0359bcb None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:02.774462 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-6dc1e820-98c8-4459-85b3-43fad0359bcb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:02.774462 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:02.774462 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:02.774462 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:02.774462 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:02.774462 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:02.774462 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:02.774462 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:02.774462 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:02.774462 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:02.774462 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:02.774462 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:02.774462 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:15:02.774462 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:15:02.774462 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:15:02.775333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:15:02.775333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:02.775333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:02.775333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:02.775333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:02.775333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:02.775333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:02.775333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:02.775333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:02.775333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:02.775333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:02.775333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:02.775333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:02.775333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:02.775333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:02.775333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:02.775333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:02.776221 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:15:02.776221 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:15:02.776221 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:15:02.776221 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:15:02.776221 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:15:02.776221 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:15:02.776221 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:02.776221 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:02.776221 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 300/601] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:15:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:02.777372 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1e820-98c8-4459-85b3-43fad0359bcb None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:02.797687 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12688d-09f6-4184-914f-5a543cb933c9 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:02.832239 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6adf2a-008d-4dde-a3eb-3aa8e20fd379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:02.832861 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6adf2a-008d-4dde-a3eb-3aa8e20fd379 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:02.833134 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6adf2a-008d-4dde-a3eb-3aa8e20fd379 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:02.833451 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6adf2a-008d-4dde-a3eb-3aa8e20fd379 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:02.841541 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6adf2a-008d-4dde-a3eb-3aa8e20fd379 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:02.843031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6adf2a-008d-4dde-a3eb-3aa8e20fd379 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:02.854200 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-0d6adf2a-008d-4dde-a3eb-3aa8e20fd379 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:02.854200 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:02.854200 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:02.854200 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:02.854200 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:02.854200 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:02.854200 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:02.854200 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:02.854200 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:02.854200 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:02.854200 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:02.854200 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:02.854200 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:15:02.854200 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:15:02.854200 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:15:02.855076 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:15:02.855076 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:02.855076 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:02.855076 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:02.855076 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:02.855076 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:02.855076 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:02.855076 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:02.855076 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:02.855076 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:02.855076 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:02.855076 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:02.855076 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:02.855076 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:02.855076 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:02.855076 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:02.855076 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:02.856008 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:15:02.856008 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:15:02.856008 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:15:02.856008 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:15:02.856008 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:15:02.856008 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:15:02.856008 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:02.856008 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:02.860016 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 302/602] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:15:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:02.861266 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81f32b-8a13-4e7d-9fb9-e25db84f1f98 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:02.861602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81f32b-8a13-4e7d-9fb9-e25db84f1f98 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:02.861882 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81f32b-8a13-4e7d-9fb9-e25db84f1f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:02.879390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-aa81f32b-8a13-4e7d-9fb9-e25db84f1f98 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:02.880647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-aa81f32b-8a13-4e7d-9fb9-e25db84f1f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:02.889242 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:02.895437 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1e820-98c8-4459-85b3-43fad0359bcb None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:02.917215 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:02.932656 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2fb125-0fcf-4929-8a0c-47e204f018e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:02.933431 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fb125-0fcf-4929-8a0c-47e204f018e7 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:02.933556 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fb125-0fcf-4929-8a0c-47e204f018e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:02.933814 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fb125-0fcf-4929-8a0c-47e204f018e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:02.934577 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2fb125-0fcf-4929-8a0c-47e204f018e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:02.934832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2fb125-0fcf-4929-8a0c-47e204f018e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:02.936285 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2fb125-0fcf-4929-8a0c-47e204f018e7 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:02.948886 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 301/603] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:15:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:02.953078 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2fb125-0fcf-4929-8a0c-47e204f018e7 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:02.954931 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 303/604] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:15:02.955846 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa81f32b-8a13-4e7d-9fb9-e25db84f1f98 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:03.063121 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa81f32b-8a13-4e7d-9fb9-e25db84f1f98 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:03.089550 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2fb125-0fcf-4929-8a0c-47e204f018e7 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:03.103650 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824dea6-ec5e-46eb-84b9-b43f72b791a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:03.104796 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824dea6-ec5e-46eb-84b9-b43f72b791a5 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:03.105276 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824dea6-ec5e-46eb-84b9-b43f72b791a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:03.106036 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824dea6-ec5e-46eb-84b9-b43f72b791a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:03.107446 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824dea6-ec5e-46eb-84b9-b43f72b791a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:03.108183 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824dea6-ec5e-46eb-84b9-b43f72b791a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:03.110463 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824dea6-ec5e-46eb-84b9-b43f72b791a5 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:03.126845 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 304/605] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:15:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:03.127990 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f4005-db21-46b4-a6ad-4b92ddce2093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:03.128322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824dea6-ec5e-46eb-84b9-b43f72b791a5 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:03.128734 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f4005-db21-46b4-a6ad-4b92ddce2093 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:03.129079 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f4005-db21-46b4-a6ad-4b92ddce2093 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:03.129439 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f4005-db21-46b4-a6ad-4b92ddce2093 None admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:03.131964 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f4005-db21-46b4-a6ad-4b92ddce2093 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:03.131964 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f4005-db21-46b4-a6ad-4b92ddce2093 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:03.132468 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f4005-db21-46b4-a6ad-4b92ddce2093 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:03.136592 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:03.155024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 302/606] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:15:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:03.155024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f4005-db21-46b4-a6ad-4b92ddce2093 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:03.204653 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:03.249053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824dea6-ec5e-46eb-84b9-b43f72b791a5 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:03.268757 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f4005-db21-46b4-a6ad-4b92ddce2093 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:03.287069 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b37e9-b2e9-484c-95f1-6ad309c7a703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:03.287788 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b37e9-b2e9-484c-95f1-6ad309c7a703 None admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:03.288099 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b37e9-b2e9-484c-95f1-6ad309c7a703 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:03.288371 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b37e9-b2e9-484c-95f1-6ad309c7a703 None admin] PATH_INFO: `/v3/projects/9f4373dfb5f9425eb73669e24d18a08f/users/4f1bf93cff6c46e0801bb827203cb5ad/roles/bdf71ef6ab10430e8c356cd4d952ff5c` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:03.302971 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a2501-b3e0-428e-93af-0fbe58c02c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:03.303628 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a2501-b3e0-428e-93af-0fbe58c02c0c None admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:03.303902 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a2501-b3e0-428e-93af-0fbe58c02c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:03.304215 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a2501-b3e0-428e-93af-0fbe58c02c0c None admin] PATH_INFO: `/v3/projects/9f4373dfb5f9425eb73669e24d18a08f/users/2db57fe3915446acb57b4f84ca353379/roles/bdf71ef6ab10430e8c356cd4d952ff5c` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:03.306628 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735b37e9-b2e9-484c-95f1-6ad309c7a703 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f4373dfb5f9425eb73669e24d18a08f, user_id=4f1bf93cff6c46e0801bb827203cb5ad, role_id=bdf71ef6ab10430e8c356cd4d952ff5c)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:03.309065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735b37e9-b2e9-484c-95f1-6ad309c7a703 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:03.319187 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a2501-b3e0-428e-93af-0fbe58c02c0c None admin] RBAC: Authorizing `identity:create_grant(project_id=9f4373dfb5f9425eb73669e24d18a08f, user_id=2db57fe3915446acb57b4f84ca353379, role_id=bdf71ef6ab10430e8c356cd4d952ff5c)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:03.320790 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a2501-b3e0-428e-93af-0fbe58c02c0c None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:03.325013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 305/607] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Aug 23 04:15:03 2022] PUT /identity/v3/projects/9f4373dfb5f9425eb73669e24d18a08f/users/4f1bf93cff6c46e0801bb827203cb5ad/roles/bdf71ef6ab10430e8c356cd4d952ff5c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:15:03.325927 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b37e9-b2e9-484c-95f1-6ad309c7a703 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:03.351596 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 303/608] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Aug 23 04:15:03 2022] PUT /identity/v3/projects/9f4373dfb5f9425eb73669e24d18a08f/users/2db57fe3915446acb57b4f84ca353379/roles/bdf71ef6ab10430e8c356cd4d952ff5c => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:15:03.352948 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a2501-b3e0-428e-93af-0fbe58c02c0c None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:03.393161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:03.455237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b37e9-b2e9-484c-95f1-6ad309c7a703 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:03.490188 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:03.498216 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaafb5f-1843-4a8b-9857-2ff256e28246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:03.498908 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaafb5f-1843-4a8b-9857-2ff256e28246 None admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:03.499193 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaafb5f-1843-4a8b-9857-2ff256e28246 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:03.499468 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaafb5f-1843-4a8b-9857-2ff256e28246 None admin] PATH_INFO: `/v3/projects/9f4373dfb5f9425eb73669e24d18a08f/users/fdca9c48e5d740f6bda8c48072b54c0b/roles/bdf71ef6ab10430e8c356cd4d952ff5c` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:03.503396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a2501-b3e0-428e-93af-0fbe58c02c0c None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:03.516342 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaafb5f-1843-4a8b-9857-2ff256e28246 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f4373dfb5f9425eb73669e24d18a08f, user_id=fdca9c48e5d740f6bda8c48072b54c0b, role_id=bdf71ef6ab10430e8c356cd4d952ff5c)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:03.517875 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaafb5f-1843-4a8b-9857-2ff256e28246 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:03.555663 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715a850-bfad-4f73-949d-b246324b1430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:03.557788 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715a850-bfad-4f73-949d-b246324b1430 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:03.559187 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715a850-bfad-4f73-949d-b246324b1430 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:03.559688 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715a850-bfad-4f73-949d-b246324b1430 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:03.560238 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 306/609] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Aug 23 04:15:03 2022] PUT /identity/v3/projects/9f4373dfb5f9425eb73669e24d18a08f/users/fdca9c48e5d740f6bda8c48072b54c0b/roles/bdf71ef6ab10430e8c356cd4d952ff5c => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:15:03.574939 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9715a850-bfad-4f73-949d-b246324b1430 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:15:03.580489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-9715a850-bfad-4f73-949d-b246324b1430 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:15:03.580489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:03.580489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:03.580489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:03.580489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:03.580489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:03.580489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:03.580489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:03.580489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:03.580489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:03.580489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:03.580489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:03.580489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:15:03.580489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:15:03.580489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:15:03.580489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:15:03.581498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:15:03.581498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:15:03.581498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:15:03.581498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:15:03.581498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:03.581498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:03.581498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:03.581498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:03.581498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:03.581498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:03.581498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:03.581498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:03.581498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:03.581498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:03.581498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:03.581498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:03.582354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:03.582354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:03.582354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:03.582354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:03.582354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:15:03.582354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:15:03.582354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:15:03.582354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:15:03.582354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:15:03.582354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:15:03.582354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:15:03.582354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:03.585868 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 304/610] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Aug 23 04:15:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:03.592844 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaafb5f-1843-4a8b-9857-2ff256e28246 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:03.720725 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaafb5f-1843-4a8b-9857-2ff256e28246 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:03.722411 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:03.761845 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a6de7c-bb99-462c-852e-2421319d2566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:03.762575 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6de7c-bb99-462c-852e-2421319d2566 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:03.762925 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6de7c-bb99-462c-852e-2421319d2566 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:03.763265 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6de7c-bb99-462c-852e-2421319d2566 None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:03.765373 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a6de7c-bb99-462c-852e-2421319d2566 None admin] RBAC: Adding query filter params (name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:03.765373 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a6de7c-bb99-462c-852e-2421319d2566 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:03.765934 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a6de7c-bb99-462c-852e-2421319d2566 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:03.775498 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 307/611] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Aug 23 04:15:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:03.780973 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715a850-bfad-4f73-949d-b246324b1430 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:03.861779 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03bcd2-7cac-4d4c-86dc-abda82679662 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:03.862339 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03bcd2-7cac-4d4c-86dc-abda82679662 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:03.862791 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03bcd2-7cac-4d4c-86dc-abda82679662 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:03.864414 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 308/612] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:15:03.877063 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4201884-e566-4bb0-83f1-28c12ec8082a None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:03.877666 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4201884-e566-4bb0-83f1-28c12ec8082a None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:03.878508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4201884-e566-4bb0-83f1-28c12ec8082a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:03.909611 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-f4201884-e566-4bb0-83f1-28c12ec8082a None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:03.913271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-f4201884-e566-4bb0-83f1-28c12ec8082a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:03.921421 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715a850-bfad-4f73-949d-b246324b1430 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:03.971150 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f5c27d-0b1c-41ad-b179-5065a8e46d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:03.972217 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5c27d-0b1c-41ad-b179-5065a8e46d38 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:03.972700 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5c27d-0b1c-41ad-b179-5065a8e46d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:03.973169 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5c27d-0b1c-41ad-b179-5065a8e46d38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:03.987978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f5c27d-0b1c-41ad-b179-5065a8e46d38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:03.989889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f5c27d-0b1c-41ad-b179-5065a8e46d38 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:04.007104 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-a0f5c27d-0b1c-41ad-b179-5065a8e46d38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:04.007104 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:04.007104 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:04.007104 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:04.007104 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:04.007104 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:04.007104 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:04.007104 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:04.007104 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:04.007104 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:04.007104 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:04.007104 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:04.007104 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:04.007104 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:04.007104 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:04.007104 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:04.008116 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:04.008116 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:04.008116 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:04.008116 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:04.008116 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:04.008116 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:04.008116 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:04.008116 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:04.008116 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:04.008116 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:04.008116 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:04.008116 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:04.008116 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:04.008116 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:04.008116 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:04.008116 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:04.008116 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:04.008965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:04.008965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:04.008965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:04.011235 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 305/613] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:04.016746 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f5c27d-0b1c-41ad-b179-5065a8e46d38 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:04.021386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:04.051367 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 309/614] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:15:04.145750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f5c27d-0b1c-41ad-b179-5065a8e46d38 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:04.182185 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7abc9-3d03-4898-8649-471265391ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:04.183234 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7abc9-3d03-4898-8649-471265391ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:04.183714 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7abc9-3d03-4898-8649-471265391ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:04.184211 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7abc9-3d03-4898-8649-471265391ad3 None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:04.185054 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7abc9-3d03-4898-8649-471265391ad3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:04.185568 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7abc9-3d03-4898-8649-471265391ad3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:04.187160 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7abc9-3d03-4898-8649-471265391ad3 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:04.200798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 306/615] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:04.209034 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4201884-e566-4bb0-83f1-28c12ec8082a None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:04.322802 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:04.330853 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4201884-e566-4bb0-83f1-28c12ec8082a None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:04.372929 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0637c4-1e4e-4122-a62d-bff5d823b0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:04.373726 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0637c4-1e4e-4122-a62d-bff5d823b0a5 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:04.374107 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0637c4-1e4e-4122-a62d-bff5d823b0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:04.374476 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0637c4-1e4e-4122-a62d-bff5d823b0a5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:04.381997 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0637c4-1e4e-4122-a62d-bff5d823b0a5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:04.383631 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0637c4-1e4e-4122-a62d-bff5d823b0a5 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:04.396051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-db0637c4-1e4e-4122-a62d-bff5d823b0a5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 04:15:04.396051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:04.396051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:04.396051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:04.396051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:04.396051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:04.396051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:04.396051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:04.396051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:04.396051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:04.396051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:04.396051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:04.396051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:15:04.396051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:15:04.396051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:15:04.396051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:15:04.397225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:04.397225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:04.397225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:15:04.397225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:04.397225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:15:04.397225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:04.397225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:04.397225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:04.397225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:04.397225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:04.397225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:04.397225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:04.397225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:04.397225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:04.397225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:04.397225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:04.397225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:04.397225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:04.398389 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:04.398389 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:04.398389 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:15:04.398389 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:15:04.398389 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:15:04.398389 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:15:04.398389 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:15:04.398389 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:15:04.398389 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 04:15:04.398389 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:04.400143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 310/616] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:15:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:04.405668 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7abc9-3d03-4898-8649-471265391ad3 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:04.461346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0637c4-1e4e-4122-a62d-bff5d823b0a5 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:04.491051 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:04.534994 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7abc9-3d03-4898-8649-471265391ad3 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:04.570911 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6796dcd7-e895-43ff-beb5-f4ce2d676c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:04.571723 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796dcd7-e895-43ff-beb5-f4ce2d676c48 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:04.572142 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796dcd7-e895-43ff-beb5-f4ce2d676c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:04.572512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796dcd7-e895-43ff-beb5-f4ce2d676c48 None admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:04.573346 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6796dcd7-e895-43ff-beb5-f4ce2d676c48 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:04.573752 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6796dcd7-e895-43ff-beb5-f4ce2d676c48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:04.575291 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6796dcd7-e895-43ff-beb5-f4ce2d676c48 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:04.603939 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 307/617] 10.222.0.18 () {56 vars in 1135 bytes} [Tue Aug 23 04:15:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:04.603939 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6796dcd7-e895-43ff-beb5-f4ce2d676c48 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:04.690664 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0637c4-1e4e-4122-a62d-bff5d823b0a5 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:04.706710 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6796dcd7-e895-43ff-beb5-f4ce2d676c48 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:04.728772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:04.744900 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d510e-f32f-463f-89a5-3cee609d49b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:04.745546 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d510e-f32f-463f-89a5-3cee609d49b3 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:04.745826 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d510e-f32f-463f-89a5-3cee609d49b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:04.746099 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d510e-f32f-463f-89a5-3cee609d49b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:04.757684 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d510e-f32f-463f-89a5-3cee609d49b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:04.759152 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d510e-f32f-463f-89a5-3cee609d49b3 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:04.774739 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-313d510e-f32f-463f-89a5-3cee609d49b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:04.774739 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:04.774739 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:04.774739 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:04.774739 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:04.774739 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:04.774739 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:04.774739 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:04.774739 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:04.774739 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:04.774739 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:04.774739 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:04.774739 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:04.774739 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:04.774739 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:04.774739 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:04.775690 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:04.775690 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:04.775690 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:04.775690 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:04.775690 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:04.775690 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:04.775690 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:04.775690 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:04.775690 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:04.775690 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:04.775690 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:04.775690 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:04.775690 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:04.775690 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:04.775690 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:04.775690 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:04.775690 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:04.776663 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:04.776663 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:04.776663 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:04.780299 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 308/618] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:15:04.782134 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d510e-f32f-463f-89a5-3cee609d49b3 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:04.792813 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b2d32-fbd3-4db5-98e2-927e0d2bf802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:04.793495 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b2d32-fbd3-4db5-98e2-927e0d2bf802 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:04.793764 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b2d32-fbd3-4db5-98e2-927e0d2bf802 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:04.794030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b2d32-fbd3-4db5-98e2-927e0d2bf802 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:04.828844 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b2d32-fbd3-4db5-98e2-927e0d2bf802 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:15:04.834023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-c33b2d32-fbd3-4db5-98e2-927e0d2bf802 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:15:04.834023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:04.834023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:04.834023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:04.834023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:04.834023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:04.834023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:04.834023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:04.834023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:04.834023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:04.834023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:04.834023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:04.834023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:15:04.834023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:15:04.834023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:15:04.834023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:15:04.835283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:15:04.835283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:15:04.835283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:15:04.835283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:15:04.835283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:04.835283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:04.835283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:04.835283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:04.835283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:04.835283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:04.835283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:04.835283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:04.835283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:04.835283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:04.835283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:04.835283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:04.836147 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:04.836147 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:04.836147 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:04.836147 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:04.836147 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:15:04.836147 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:15:04.836147 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:15:04.836147 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:15:04.836147 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:15:04.836147 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:15:04.836147 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:15:04.836147 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:04.844406 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 311/619] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Aug 23 04:15:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:04.850386 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b2d32-fbd3-4db5-98e2-927e0d2bf802 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:04.889050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:04.895335 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d510e-f32f-463f-89a5-3cee609d49b3 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:04.931725 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba66339-47cc-4452-87f9-e13bd5cd95c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:04.932365 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba66339-47cc-4452-87f9-e13bd5cd95c9 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:04.932704 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba66339-47cc-4452-87f9-e13bd5cd95c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:04.932992 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba66339-47cc-4452-87f9-e13bd5cd95c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:04.933705 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba66339-47cc-4452-87f9-e13bd5cd95c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:04.934011 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba66339-47cc-4452-87f9-e13bd5cd95c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:04.935385 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba66339-47cc-4452-87f9-e13bd5cd95c9 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:04.952925 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 309/620] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:04.954833 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba66339-47cc-4452-87f9-e13bd5cd95c9 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:04.988544 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:05.065652 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b2d32-fbd3-4db5-98e2-927e0d2bf802 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:05.089961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba66339-47cc-4452-87f9-e13bd5cd95c9 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:05.128816 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab355c0a-9311-4386-bc35-88487e012298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:05.129442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab355c0a-9311-4386-bc35-88487e012298 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:05.132163 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab355c0a-9311-4386-bc35-88487e012298 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:05.132471 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab355c0a-9311-4386-bc35-88487e012298 None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:05.133170 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab355c0a-9311-4386-bc35-88487e012298 None admin] RBAC: Adding query filter params (name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:05.133478 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab355c0a-9311-4386-bc35-88487e012298 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:05.137706 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab355c0a-9311-4386-bc35-88487e012298 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:05.157085 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 312/621] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Aug 23 04:15:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:05.159180 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b3220-00f5-4c72-af26-bf66e072c2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:05.159828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b3220-00f5-4c72-af26-bf66e072c2a2 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:05.160132 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b3220-00f5-4c72-af26-bf66e072c2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:05.160404 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b3220-00f5-4c72-af26-bf66e072c2a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:05.163843 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab355c0a-9311-4386-bc35-88487e012298 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:05.177828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b3220-00f5-4c72-af26-bf66e072c2a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:05.179374 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b3220-00f5-4c72-af26-bf66e072c2a2 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:05.198165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-4e4b3220-00f5-4c72-af26-bf66e072c2a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:05.198165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:05.198165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:05.198165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:05.198165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:05.198165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:05.198165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:05.198165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:05.198165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:05.198165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:05.198165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:05.198165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:05.198165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:15:05.198165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:15:05.198165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:15:05.199098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:15:05.199098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:05.199098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:05.199098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:05.199098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:05.199098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:05.199098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:05.199098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:05.199098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:05.199098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:05.199098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:05.199098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:05.199098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:05.199098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:05.199098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:05.199098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:05.199098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:05.199955 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:15:05.199955 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:15:05.199955 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:15:05.199955 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:15:05.199955 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:15:05.199955 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:15:05.199955 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:05.199955 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:05.205559 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 310/622] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:15:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:05.210748 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b3220-00f5-4c72-af26-bf66e072c2a2 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:05.336402 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:05.346709 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b3220-00f5-4c72-af26-bf66e072c2a2 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:05.350451 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:05.400170 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab355c0a-9311-4386-bc35-88487e012298 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:05.403502 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1761bb85-0fbf-4a0f-a01a-a9e7064c109c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:05.404146 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761bb85-0fbf-4a0f-a01a-a9e7064c109c None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:05.404416 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761bb85-0fbf-4a0f-a01a-a9e7064c109c None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:05.404679 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761bb85-0fbf-4a0f-a01a-a9e7064c109c None admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:05.405372 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1761bb85-0fbf-4a0f-a01a-a9e7064c109c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:05.405668 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1761bb85-0fbf-4a0f-a01a-a9e7064c109c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:05.407110 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1761bb85-0fbf-4a0f-a01a-a9e7064c109c None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:05.435961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 311/623] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:15:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:05.442883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1761bb85-0fbf-4a0f-a01a-a9e7064c109c None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:05.482387 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051a237-7589-409d-ac94-2580876a0a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:05.483031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051a237-7589-409d-ac94-2580876a0a7c None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:05.483355 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051a237-7589-409d-ac94-2580876a0a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:05.483625 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051a237-7589-409d-ac94-2580876a0a7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:05.498191 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b051a237-7589-409d-ac94-2580876a0a7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:05.499652 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b051a237-7589-409d-ac94-2580876a0a7c None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:05.533170 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-b051a237-7589-409d-ac94-2580876a0a7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:05.533170 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:05.533170 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:05.533170 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:05.533170 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:05.533170 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:05.533170 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:05.533170 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:05.533170 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:05.533170 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:05.533170 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:05.533170 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:05.533170 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:05.533170 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:05.533170 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:05.533170 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:05.534159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:05.534159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:05.534159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:05.534159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:05.534159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:05.534159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:05.534159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:05.534159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:05.534159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:05.534159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:05.534159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:05.534159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:05.534159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:05.534159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:05.534159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:05.534159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:05.534159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:05.535028 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:05.535028 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:05.535028 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:05.537425 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 313/624] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:15:05.542057 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051a237-7589-409d-ac94-2580876a0a7c None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:05.619061 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1761bb85-0fbf-4a0f-a01a-a9e7064c109c None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:05.654420 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59270710-524b-4940-b147-43b6091cefad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:05.655128 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59270710-524b-4940-b147-43b6091cefad None admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:05.655405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59270710-524b-4940-b147-43b6091cefad None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:05.655676 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59270710-524b-4940-b147-43b6091cefad None admin] PATH_INFO: `/v3/projects/9f4373dfb5f9425eb73669e24d18a08f/users/3f1418fd211a448da1a50175dce13b32/roles/bdf71ef6ab10430e8c356cd4d952ff5c` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:05.689631 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59270710-524b-4940-b147-43b6091cefad None admin] RBAC: Authorizing `identity:create_grant(project_id=9f4373dfb5f9425eb73669e24d18a08f, user_id=3f1418fd211a448da1a50175dce13b32, role_id=bdf71ef6ab10430e8c356cd4d952ff5c)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:05.690980 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59270710-524b-4940-b147-43b6091cefad None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:05.695814 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051a237-7589-409d-ac94-2580876a0a7c None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:05.724035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 312/625] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Aug 23 04:15:05 2022] PUT /identity/v3/projects/9f4373dfb5f9425eb73669e24d18a08f/users/3f1418fd211a448da1a50175dce13b32/roles/bdf71ef6ab10430e8c356cd4d952ff5c => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:15:05.724035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd634fc-8921-46bf-8ee1-238c8b88186b None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:05.724035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd634fc-8921-46bf-8ee1-238c8b88186b None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:05.724035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd634fc-8921-46bf-8ee1-238c8b88186b None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:05.727665 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 313/626] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:05.729143 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d323b-7618-43e3-937d-905fe69e1d82 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:05.729426 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d323b-7618-43e3-937d-905fe69e1d82 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:05.729700 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d323b-7618-43e3-937d-905fe69e1d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:05.732622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:05.737679 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:05.752596 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-8e1d323b-7618-43e3-937d-905fe69e1d82 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:05.753838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-8e1d323b-7618-43e3-937d-905fe69e1d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:05.754679 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d05592-0173-4cd7-85e7-da8b4464e88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:05.755551 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d05592-0173-4cd7-85e7-da8b4464e88b None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:05.756299 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d05592-0173-4cd7-85e7-da8b4464e88b None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:05.756821 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d05592-0173-4cd7-85e7-da8b4464e88b None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:05.757655 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d05592-0173-4cd7-85e7-da8b4464e88b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:05.758166 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d05592-0173-4cd7-85e7-da8b4464e88b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:05.759772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d05592-0173-4cd7-85e7-da8b4464e88b None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:05.790434 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 314/627] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:05.794622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d05592-0173-4cd7-85e7-da8b4464e88b None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:05.843923 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 314/628] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:15:05.853539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0837fc11-cbdd-4775-aadd-8176040e6db3 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:05.853848 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0837fc11-cbdd-4775-aadd-8176040e6db3 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:05.854209 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0837fc11-cbdd-4775-aadd-8176040e6db3 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:05.859800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 315/629] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:05.861469 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771cc671-2514-48fa-a7da-23daf0e5f132 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:05.861825 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771cc671-2514-48fa-a7da-23daf0e5f132 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:05.862141 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771cc671-2514-48fa-a7da-23daf0e5f132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:05.897331 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-771cc671-2514-48fa-a7da-23daf0e5f132 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:05.898788 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-771cc671-2514-48fa-a7da-23daf0e5f132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:05.919688 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d05592-0173-4cd7-85e7-da8b4464e88b None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:05.970795 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98efe313-7e9d-48d5-980b-108648e0b2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:05.971432 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efe313-7e9d-48d5-980b-108648e0b2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:05.971792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efe313-7e9d-48d5-980b-108648e0b2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:05.971999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efe313-7e9d-48d5-980b-108648e0b2c1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:05.983843 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98efe313-7e9d-48d5-980b-108648e0b2c1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:05.985264 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98efe313-7e9d-48d5-980b-108648e0b2c1 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:05.993898 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 316/630] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 04:15:05.995265 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229fbeec-4a52-43c3-b7ab-03aad1072661 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:05.995620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229fbeec-4a52-43c3-b7ab-03aad1072661 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:05.996032 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229fbeec-4a52-43c3-b7ab-03aad1072661 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:05.999961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 317/631] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:06.004406 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e842031-69d4-47a5-8b81-1ba5b26abe91 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:06.004781 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e842031-69d4-47a5-8b81-1ba5b26abe91 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:06.005187 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e842031-69d4-47a5-8b81-1ba5b26abe91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:06.006872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-98efe313-7e9d-48d5-980b-108648e0b2c1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 04:15:06.006872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:06.006872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:06.006872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:06.006872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:06.006872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:06.006872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:06.006872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:06.006872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:06.006872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:06.006872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:06.006872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:06.006872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:15:06.006872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:15:06.006872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:15:06.006872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:15:06.008026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:06.008026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:06.008026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:15:06.008026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:06.008026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:15:06.008026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:06.008026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:06.008026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:06.008026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:06.008026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:06.008026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:06.008026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:06.008026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:06.008026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:06.008026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:06.008026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:06.008026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:06.008026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:06.009062 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:06.009062 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:06.009062 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:15:06.009062 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:15:06.009062 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:15:06.009062 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:15:06.009062 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:15:06.009062 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:15:06.009062 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 04:15:06.009062 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:06.011407 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 315/632] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Aug 23 04:15:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:06.017428 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98efe313-7e9d-48d5-980b-108648e0b2c1 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:06.030791 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-5e842031-69d4-47a5-8b81-1ba5b26abe91 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:06.032160 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-5e842031-69d4-47a5-8b81-1ba5b26abe91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:06.090169 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:06.124664 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:06.145111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 318/633] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:15:06.169872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e842031-69d4-47a5-8b81-1ba5b26abe91 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:06.171291 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98efe313-7e9d-48d5-980b-108648e0b2c1 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:06.206057 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8810bd8-ce35-4a35-8de6-60710d579a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:06.206683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8810bd8-ce35-4a35-8de6-60710d579a89 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:06.206967 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8810bd8-ce35-4a35-8de6-60710d579a89 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:06.207244 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8810bd8-ce35-4a35-8de6-60710d579a89 None admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:06.208008 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8810bd8-ce35-4a35-8de6-60710d579a89 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:06.208319 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8810bd8-ce35-4a35-8de6-60710d579a89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:06.209851 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8810bd8-ce35-4a35-8de6-60710d579a89 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:06.228663 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 316/634] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Aug 23 04:15:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:06.234541 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8810bd8-ce35-4a35-8de6-60710d579a89 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:06.334423 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e842031-69d4-47a5-8b81-1ba5b26abe91 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:06.371618 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975837e-53b7-4b98-8159-fc1d3fcb8e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:06.372296 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975837e-53b7-4b98-8159-fc1d3fcb8e08 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:06.372623 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975837e-53b7-4b98-8159-fc1d3fcb8e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:06.372899 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975837e-53b7-4b98-8159-fc1d3fcb8e08 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:06.385164 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975837e-53b7-4b98-8159-fc1d3fcb8e08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:15:06.390238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-1975837e-53b7-4b98-8159-fc1d3fcb8e08 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:15:06.390238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:06.390238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:06.390238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:06.390238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:06.390238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:06.390238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:06.390238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:06.390238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:06.390238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:06.390238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:06.390238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:06.390238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:15:06.390238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:15:06.390238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:15:06.390238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:15:06.391118 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:15:06.391118 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:15:06.391118 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:15:06.391118 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:15:06.391118 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:06.391118 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:06.391118 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:06.391118 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:06.391118 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:06.391118 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:06.391118 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:06.391118 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:06.391118 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:06.391118 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:06.391118 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:06.391118 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:06.391908 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:06.391908 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:06.391908 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:06.391908 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:06.391908 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:15:06.391908 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:15:06.391908 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:15:06.391908 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:15:06.391908 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:15:06.391908 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:15:06.391908 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:15:06.391908 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:06.391908 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 319/635] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Aug 23 04:15:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:15:06.392606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975837e-53b7-4b98-8159-fc1d3fcb8e08 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:06.402601 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8810bd8-ce35-4a35-8de6-60710d579a89 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:06.410366 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:06.440139 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1073d859-64b2-4bab-acc2-e1da8265079b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:06.440753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1073d859-64b2-4bab-acc2-e1da8265079b None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:06.441030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1073d859-64b2-4bab-acc2-e1da8265079b None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:06.441298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1073d859-64b2-4bab-acc2-e1da8265079b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:06.462324 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:06.517000 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1073d859-64b2-4bab-acc2-e1da8265079b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:06.518520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1073d859-64b2-4bab-acc2-e1da8265079b None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:06.520301 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975837e-53b7-4b98-8159-fc1d3fcb8e08 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:06.557615 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea01c63-bca3-4a66-b5ab-88efa91634a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:06.558293 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea01c63-bca3-4a66-b5ab-88efa91634a4 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:06.558575 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea01c63-bca3-4a66-b5ab-88efa91634a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:06.558842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea01c63-bca3-4a66-b5ab-88efa91634a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:06.570273 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea01c63-bca3-4a66-b5ab-88efa91634a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:15:06.575286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-fea01c63-bca3-4a66-b5ab-88efa91634a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:15:06.575286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:06.575286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:06.575286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:06.575286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:06.575286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:06.575286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:06.575286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:06.575286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:06.575286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:06.575286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:06.575286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:06.575286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:15:06.575286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:15:06.575286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:15:06.575286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:15:06.576207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:15:06.576207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:15:06.576207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:15:06.576207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:15:06.576207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:06.576207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:06.576207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:06.576207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:06.576207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:06.576207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:06.576207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:06.576207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:06.576207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:06.576207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:06.576207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:06.576207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:06.577018 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:06.577018 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:06.577018 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:06.577018 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:06.577018 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:15:06.577018 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:15:06.577018 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:15:06.577018 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:15:06.577018 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:15:06.577018 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:15:06.577018 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:15:06.577018 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:06.577018 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 320/636] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Aug 23 04:15:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:06.577711 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea01c63-bca3-4a66-b5ab-88efa91634a4 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:06.628815 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 317/637] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:15:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:15:06.629541 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1073d859-64b2-4bab-acc2-e1da8265079b None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:06.640024 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:06.686581 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea01c63-bca3-4a66-b5ab-88efa91634a4 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:06.699693 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:06.722649 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e57d6-e4a2-4704-bff5-3f9d267e1141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:06.723366 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e57d6-e4a2-4704-bff5-3f9d267e1141 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:06.723709 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e57d6-e4a2-4704-bff5-3f9d267e1141 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:06.724075 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e57d6-e4a2-4704-bff5-3f9d267e1141 None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:06.725096 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e57d6-e4a2-4704-bff5-3f9d267e1141 None admin] RBAC: Adding query filter params (name=service) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:06.725372 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e57d6-e4a2-4704-bff5-3f9d267e1141 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:06.726760 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e57d6-e4a2-4704-bff5-3f9d267e1141 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:06.733970 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1073d859-64b2-4bab-acc2-e1da8265079b None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:06.739673 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 321/638] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Aug 23 04:15:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:06.739673 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e57d6-e4a2-4704-bff5-3f9d267e1141 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:06.768440 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7637859-271d-408d-8c1c-59778ed161f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:06.768996 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7637859-271d-408d-8c1c-59778ed161f2 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:06.769274 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7637859-271d-408d-8c1c-59778ed161f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:06.769543 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7637859-271d-408d-8c1c-59778ed161f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:06.780056 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7637859-271d-408d-8c1c-59778ed161f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:15:06.785206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-c7637859-271d-408d-8c1c-59778ed161f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:15:06.785206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:06.785206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:06.785206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:06.785206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:06.785206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:06.785206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:06.785206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:06.785206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:06.785206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:06.785206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:06.785206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:06.785206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:15:06.785206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:15:06.785206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:15:06.785206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:15:06.786112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:15:06.786112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:15:06.786112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:15:06.786112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:15:06.786112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:06.786112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:06.786112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:06.786112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:06.786112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:06.786112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:06.786112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:06.786112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:06.786112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:06.786112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:06.786112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:06.786112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:06.787016 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:06.787016 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:06.787016 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:06.787016 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:06.787016 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:15:06.787016 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:15:06.787016 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:15:06.787016 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:15:06.787016 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:15:06.787016 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:15:06.787016 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:15:06.787016 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:06.787016 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 318/639] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Aug 23 04:15:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:06.787810 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7637859-271d-408d-8c1c-59778ed161f2 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:06.858893 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:06.862588 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e57d6-e4a2-4704-bff5-3f9d267e1141 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:06.896855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7637859-271d-408d-8c1c-59778ed161f2 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:06.907815 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3844a13-aacd-4ced-8bfe-0ae917d6f41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:06.908610 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3844a13-aacd-4ced-8bfe-0ae917d6f41e None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:06.908744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3844a13-aacd-4ced-8bfe-0ae917d6f41e None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:06.909006 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3844a13-aacd-4ced-8bfe-0ae917d6f41e None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:06.909720 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3844a13-aacd-4ced-8bfe-0ae917d6f41e None admin] RBAC: Adding query filter params (name=service) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:06.909974 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3844a13-aacd-4ced-8bfe-0ae917d6f41e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:06.911380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3844a13-aacd-4ced-8bfe-0ae917d6f41e None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:06.925710 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 322/640] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Aug 23 04:15:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:06.925710 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3844a13-aacd-4ced-8bfe-0ae917d6f41e None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:06.941507 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87876070-6a4f-4c6f-8c41-6b57347d2f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:06.942158 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87876070-6a4f-4c6f-8c41-6b57347d2f55 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:06.942536 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87876070-6a4f-4c6f-8c41-6b57347d2f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:06.942815 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87876070-6a4f-4c6f-8c41-6b57347d2f55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:06.955091 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87876070-6a4f-4c6f-8c41-6b57347d2f55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:06.956599 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87876070-6a4f-4c6f-8c41-6b57347d2f55 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:06.962247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:06.973172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-87876070-6a4f-4c6f-8c41-6b57347d2f55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:06.973172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:06.973172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:06.973172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:06.973172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:06.973172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:06.973172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:06.973172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:06.973172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:06.973172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:06.973172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:06.973172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:06.973172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:06.973172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:06.973172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:06.973172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:06.974300 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:06.974300 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:06.974300 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:06.974300 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:06.974300 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:06.974300 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:06.974300 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:06.974300 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:06.974300 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:06.974300 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:06.974300 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:06.974300 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:06.974300 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:06.974300 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:06.974300 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:06.974300 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:06.974300 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:06.975205 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:06.975205 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:06.975205 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:06.975205 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 319/641] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:06.976012 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87876070-6a4f-4c6f-8c41-6b57347d2f55 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:07.042243 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3844a13-aacd-4ced-8bfe-0ae917d6f41e None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:07.079790 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d7b3a-56b9-4ca7-99a7-10f09742aadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:07.080539 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87876070-6a4f-4c6f-8c41-6b57347d2f55 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:07.081019 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d7b3a-56b9-4ca7-99a7-10f09742aadc None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:07.081579 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d7b3a-56b9-4ca7-99a7-10f09742aadc None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:07.082130 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d7b3a-56b9-4ca7-99a7-10f09742aadc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:07.095007 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5d7b3a-56b9-4ca7-99a7-10f09742aadc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:07.096627 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5d7b3a-56b9-4ca7-99a7-10f09742aadc None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:07.098440 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:07.112872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-4a5d7b3a-56b9-4ca7-99a7-10f09742aadc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:07.112872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:07.112872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:07.112872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:07.112872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:07.112872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:07.112872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:07.112872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:07.112872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:07.112872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:07.112872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:07.112872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:07.112872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:07.112872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:07.112872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:07.112872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:07.113880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:07.113880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:07.113880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:07.113880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:07.113880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:07.113880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:07.113880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:07.113880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:07.113880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:07.113880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:07.113880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:07.113880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:07.113880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:07.113880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:07.113880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:07.113880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:07.113880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:07.114769 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:07.114769 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:07.114769 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:07.117064 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430bb88-c740-4ab6-bab3-2a40fdf867e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:07.117864 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430bb88-c740-4ab6-bab3-2a40fdf867e0 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:07.118137 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430bb88-c740-4ab6-bab3-2a40fdf867e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:07.118398 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430bb88-c740-4ab6-bab3-2a40fdf867e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:07.119026 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 323/642] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:07.119458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430bb88-c740-4ab6-bab3-2a40fdf867e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:07.119744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d7b3a-56b9-4ca7-99a7-10f09742aadc None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:07.119837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430bb88-c740-4ab6-bab3-2a40fdf867e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:07.121199 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430bb88-c740-4ab6-bab3-2a40fdf867e0 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:07.134744 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 320/643] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Aug 23 04:15:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:07.134744 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430bb88-c740-4ab6-bab3-2a40fdf867e0 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:07.215891 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:07.230215 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d7b3a-56b9-4ca7-99a7-10f09742aadc None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:07.236699 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430bb88-c740-4ab6-bab3-2a40fdf867e0 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:07.266404 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bece6c-194e-4161-bcef-8b1eb33fc602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:07.267047 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bece6c-194e-4161-bcef-8b1eb33fc602 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:07.267324 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bece6c-194e-4161-bcef-8b1eb33fc602 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:07.267597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bece6c-194e-4161-bcef-8b1eb33fc602 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:07.279607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bece6c-194e-4161-bcef-8b1eb33fc602 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:07.281087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bece6c-194e-4161-bcef-8b1eb33fc602 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:07.287115 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b228c65-7156-4f70-8b2e-8ad1b4136d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:07.287955 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b228c65-7156-4f70-8b2e-8ad1b4136d5a None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:07.288065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b228c65-7156-4f70-8b2e-8ad1b4136d5a None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:07.288336 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b228c65-7156-4f70-8b2e-8ad1b4136d5a None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:07.288961 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b228c65-7156-4f70-8b2e-8ad1b4136d5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:07.289264 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b228c65-7156-4f70-8b2e-8ad1b4136d5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:07.290638 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b228c65-7156-4f70-8b2e-8ad1b4136d5a None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:07.297105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-c6bece6c-194e-4161-bcef-8b1eb33fc602 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:07.297105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:07.297105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:07.297105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:07.297105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:07.297105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:07.297105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:07.297105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:07.297105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:07.297105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:07.297105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:07.297105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:07.297105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:07.297105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:07.297105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:07.297105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:07.297978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:07.297978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:07.297978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:07.297978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:07.297978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:07.297978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:07.297978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:07.297978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:07.297978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:07.297978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:07.297978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:07.297978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:07.297978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:07.297978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:07.297978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:07.297978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:07.297978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:07.304710 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:07.304710 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:07.304710 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:07.304710 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 324/644] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:07.304710 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bece6c-194e-4161-bcef-8b1eb33fc602 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:07.304984 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 321/645] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:07.304984 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b228c65-7156-4f70-8b2e-8ad1b4136d5a None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:07.350331 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:07.415087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bece6c-194e-4161-bcef-8b1eb33fc602 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:07.422933 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b228c65-7156-4f70-8b2e-8ad1b4136d5a None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:07.450952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54ffe1-0932-4e3d-9b3b-fe49f5cce4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:07.451614 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54ffe1-0932-4e3d-9b3b-fe49f5cce4e8 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:07.451839 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54ffe1-0932-4e3d-9b3b-fe49f5cce4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:07.452154 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54ffe1-0932-4e3d-9b3b-fe49f5cce4e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:07.455231 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da54ffe1-0932-4e3d-9b3b-fe49f5cce4e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:07.455231 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da54ffe1-0932-4e3d-9b3b-fe49f5cce4e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:07.455231 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da54ffe1-0932-4e3d-9b3b-fe49f5cce4e8 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:07.461884 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b612c-a1e4-425b-862c-032a59907f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:07.462527 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b612c-a1e4-425b-862c-032a59907f52 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:07.462808 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b612c-a1e4-425b-862c-032a59907f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:07.463081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b612c-a1e4-425b-862c-032a59907f52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:07.464833 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:07.474955 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 325/646] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:07.474955 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54ffe1-0932-4e3d-9b3b-fe49f5cce4e8 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:07.479812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b612c-a1e4-425b-862c-032a59907f52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:07.481453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b612c-a1e4-425b-862c-032a59907f52 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:07.498164 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-541b612c-a1e4-425b-862c-032a59907f52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:07.498164 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:07.498164 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:07.498164 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:07.498164 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:07.498164 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:07.498164 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:07.498164 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:07.498164 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:07.498164 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:07.498164 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:07.498164 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:07.498164 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:07.498164 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:07.498164 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:07.498164 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:07.499143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:07.499143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:07.499143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:07.499143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:07.499143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:07.499143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:07.499143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:07.499143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:07.499143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:07.499143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:07.499143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:07.499143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:07.499143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:07.499143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:07.499143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:07.499143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:07.499143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:07.500784 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:07.500784 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:07.500784 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:07.500784 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 322/647] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:07.500784 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b612c-a1e4-425b-862c-032a59907f52 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:07.588651 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54ffe1-0932-4e3d-9b3b-fe49f5cce4e8 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:07.598415 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:07.607756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b612c-a1e4-425b-862c-032a59907f52 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:07.625205 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4689160-5b54-432b-bedc-c82de50bb62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:07.626040 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4689160-5b54-432b-bedc-c82de50bb62d None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:07.626174 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4689160-5b54-432b-bedc-c82de50bb62d None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:07.626373 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4689160-5b54-432b-bedc-c82de50bb62d None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:07.627006 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4689160-5b54-432b-bedc-c82de50bb62d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:07.627304 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4689160-5b54-432b-bedc-c82de50bb62d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:07.628690 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4689160-5b54-432b-bedc-c82de50bb62d None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:07.643348 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 326/648] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:07.644241 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4689160-5b54-432b-bedc-c82de50bb62d None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:07.655545 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6eb3c6-3f96-4888-80f7-c5322c171f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:07.656212 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6eb3c6-3f96-4888-80f7-c5322c171f31 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:07.656783 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6eb3c6-3f96-4888-80f7-c5322c171f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:07.656783 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6eb3c6-3f96-4888-80f7-c5322c171f31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:07.664852 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6eb3c6-3f96-4888-80f7-c5322c171f31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:07.666326 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6eb3c6-3f96-4888-80f7-c5322c171f31 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:07.678641 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-aa6eb3c6-3f96-4888-80f7-c5322c171f31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:07.678641 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:07.678641 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:07.678641 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:07.678641 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:07.678641 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:07.678641 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:07.678641 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:07.678641 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:07.678641 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:07.678641 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:07.678641 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:07.678641 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:15:07.678641 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:15:07.678641 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:15:07.679603 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:15:07.679603 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:07.679603 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:07.679603 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:07.679603 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:07.679603 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:07.679603 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:07.679603 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:07.679603 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:07.679603 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:07.679603 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:07.679603 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:07.679603 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:07.679603 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:07.679603 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:07.679603 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:07.679603 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:07.680589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:15:07.680589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:15:07.680589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:15:07.680589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:15:07.680589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:15:07.680589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:15:07.680589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:07.680589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:07.680589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 323/649] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:15:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:07.680589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6eb3c6-3f96-4888-80f7-c5322c171f31 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:07.714585 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:07.759668 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4689160-5b54-432b-bedc-c82de50bb62d None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:07.789151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6eb3c6-3f96-4888-80f7-c5322c171f31 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:07.795783 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cea14-a4f5-4b47-97ff-5da4a4996dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:07.796467 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cea14-a4f5-4b47-97ff-5da4a4996dd9 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:07.796738 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cea14-a4f5-4b47-97ff-5da4a4996dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:07.797005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cea14-a4f5-4b47-97ff-5da4a4996dd9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:07.804955 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748cea14-a4f5-4b47-97ff-5da4a4996dd9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:07.806393 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748cea14-a4f5-4b47-97ff-5da4a4996dd9 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:07.817309 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-748cea14-a4f5-4b47-97ff-5da4a4996dd9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 23 04:15:07.817309 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:07.817309 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:07.817309 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:07.817309 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:07.817309 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:07.817309 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:07.817309 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:07.817309 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:07.817309 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:07.817309 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:07.817309 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:07.817309 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:15:07.817309 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:15:07.817309 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:15:07.817309 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:15:07.818319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:07.818319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:07.818319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:15:07.818319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:07.818319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:15:07.818319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:07.818319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:07.818319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:07.818319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:07.818319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:07.818319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:07.818319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:07.818319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:07.818319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:07.818319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:07.818319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:07.818319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:07.818319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:07.819246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:07.819246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:07.819246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:15:07.819246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:15:07.819246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:15:07.819246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:15:07.819246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:15:07.819246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:15:07.819246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 23 04:15:07.819246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:07.822957 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 327/650] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:15:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:07.823782 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cea14-a4f5-4b47-97ff-5da4a4996dd9 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:07.823890 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8015dc94-16ad-42fa-afa7-b1fca568763a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:07.824478 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015dc94-16ad-42fa-afa7-b1fca568763a None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:07.824754 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015dc94-16ad-42fa-afa7-b1fca568763a None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:07.825032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015dc94-16ad-42fa-afa7-b1fca568763a None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:07.825676 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8015dc94-16ad-42fa-afa7-b1fca568763a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:07.825980 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8015dc94-16ad-42fa-afa7-b1fca568763a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:07.827354 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8015dc94-16ad-42fa-afa7-b1fca568763a None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:07.840235 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 324/651] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:07.840985 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8015dc94-16ad-42fa-afa7-b1fca568763a None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:07.856883 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:07.925862 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cea14-a4f5-4b47-97ff-5da4a4996dd9 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:07.945046 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8015dc94-16ad-42fa-afa7-b1fca568763a None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:07.946277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:07.964013 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ef01e-7b07-49e9-a309-79f0bb640261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:07.965005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ef01e-7b07-49e9-a309-79f0bb640261 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:07.965486 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ef01e-7b07-49e9-a309-79f0bb640261 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:07.965969 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ef01e-7b07-49e9-a309-79f0bb640261 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:07.974638 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ef01e-7b07-49e9-a309-79f0bb640261 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:07.976128 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ef01e-7b07-49e9-a309-79f0bb640261 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:07.983713 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f259fd-fb89-4011-8882-f68d9bacedb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:07.984355 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f259fd-fb89-4011-8882-f68d9bacedb4 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:07.984639 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f259fd-fb89-4011-8882-f68d9bacedb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:07.984910 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f259fd-fb89-4011-8882-f68d9bacedb4 None admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:07.985603 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f259fd-fb89-4011-8882-f68d9bacedb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:07.985909 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f259fd-fb89-4011-8882-f68d9bacedb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:07.986978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-cc2ef01e-7b07-49e9-a309-79f0bb640261 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 04:15:07.986978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:07.986978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:07.986978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:07.986978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:07.986978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:07.986978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:07.986978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:07.986978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:07.986978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:07.986978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:07.986978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:07.986978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:15:07.986978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:15:07.986978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:15:07.986978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:15:07.987867 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:07.987867 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:07.987867 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:15:07.987867 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:07.987867 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:15:07.987867 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:07.987867 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:07.987867 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:07.987867 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:07.987867 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:07.987867 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:07.987867 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:07.987867 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:07.987867 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:07.987867 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:07.987867 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:07.987867 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:07.987867 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:07.988789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:07.988789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:07.988789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:15:07.988789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:15:07.988789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:15:07.988789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:15:07.988789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:15:07.988789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:15:07.988789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 04:15:07.988789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:07.989306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f259fd-fb89-4011-8882-f68d9bacedb4 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:07.993073 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 328/652] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Aug 23 04:15:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:07.994233 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea241b8-86f6-43d3-8120-3a03d3ca10ed None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:07.994512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea241b8-86f6-43d3-8120-3a03d3ca10ed None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:07.994784 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea241b8-86f6-43d3-8120-3a03d3ca10ed None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:07.996046 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 329/653] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:07.996720 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea241b8-86f6-43d3-8120-3a03d3ca10ed None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:08.004063 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 325/654] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:15:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:08.005206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f259fd-fb89-4011-8882-f68d9bacedb4 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:08.075183 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:08.100372 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea241b8-86f6-43d3-8120-3a03d3ca10ed None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:08.106652 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f259fd-fb89-4011-8882-f68d9bacedb4 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:08.134745 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9e0f5-5228-4f29-9478-d9743066a58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:08.135347 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9e0f5-5228-4f29-9478-d9743066a58e None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:08.135700 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9e0f5-5228-4f29-9478-d9743066a58e None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:08.135883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9e0f5-5228-4f29-9478-d9743066a58e None admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:08.136644 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9e0f5-5228-4f29-9478-d9743066a58e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:08.136940 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9e0f5-5228-4f29-9478-d9743066a58e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:08.138357 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9e0f5-5228-4f29-9478-d9743066a58e None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:08.141057 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda42b6-8570-4e2b-a1d9-38bd9678dce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:08.141689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda42b6-8570-4e2b-a1d9-38bd9678dce7 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:08.141973 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda42b6-8570-4e2b-a1d9-38bd9678dce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:08.142249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda42b6-8570-4e2b-a1d9-38bd9678dce7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:08.149759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda42b6-8570-4e2b-a1d9-38bd9678dce7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:08.151206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda42b6-8570-4e2b-a1d9-38bd9678dce7 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:08.154948 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 330/655] 10.222.0.18 () {56 vars in 1137 bytes} [Tue Aug 23 04:15:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:08.155739 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9e0f5-5228-4f29-9478-d9743066a58e None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:08.162196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-1dda42b6-8570-4e2b-a1d9-38bd9678dce7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 04:15:08.162196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:08.162196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:08.162196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:08.162196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:08.162196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:08.162196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:08.162196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:08.162196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:08.162196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:08.162196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:08.162196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:08.162196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:15:08.162196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:15:08.162196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:15:08.162196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:15:08.163118 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:08.163118 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:08.163118 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:15:08.163118 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:08.163118 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:15:08.163118 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:08.163118 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:08.163118 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:08.163118 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:08.163118 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:08.163118 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:08.163118 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:08.163118 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:08.163118 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:08.163118 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:08.163118 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:08.163118 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:08.163118 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:08.164078 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:08.164078 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:08.164078 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:15:08.164078 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:15:08.164078 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:15:08.164078 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:15:08.164078 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:15:08.164078 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:15:08.164078 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 04:15:08.164078 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:08.164078 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 326/656] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:15:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:08.164665 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d8caa-55d1-47b1-ab86-34c10bcd2211 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:08.164740 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d8caa-55d1-47b1-ab86-34c10bcd2211 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:08.164984 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d8caa-55d1-47b1-ab86-34c10bcd2211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:08.183285 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-d07d8caa-55d1-47b1-ab86-34c10bcd2211 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:08.184501 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-d07d8caa-55d1-47b1-ab86-34c10bcd2211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:08.197171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:08.249964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 327/657] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:15:08.250697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d8caa-55d1-47b1-ab86-34c10bcd2211 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:08.262925 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9e0f5-5228-4f29-9478-d9743066a58e None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:08.299755 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ca4e2-8ba7-4886-bbd6-5a41fdcc36da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:08.300392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ca4e2-8ba7-4886-bbd6-5a41fdcc36da None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:08.300673 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ca4e2-8ba7-4886-bbd6-5a41fdcc36da None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:08.300941 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ca4e2-8ba7-4886-bbd6-5a41fdcc36da None admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:08.301664 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ca4e2-8ba7-4886-bbd6-5a41fdcc36da None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:08.301965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ca4e2-8ba7-4886-bbd6-5a41fdcc36da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:08.303421 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ca4e2-8ba7-4886-bbd6-5a41fdcc36da None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:08.316605 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:08.325693 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 331/658] 10.222.0.18 () {56 vars in 1139 bytes} [Tue Aug 23 04:15:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:08.325693 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ca4e2-8ba7-4886-bbd6-5a41fdcc36da None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:08.365850 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d8caa-55d1-47b1-ab86-34c10bcd2211 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:08.405264 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d40677-7452-48ed-a164-8a552697ca6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:08.406591 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d40677-7452-48ed-a164-8a552697ca6c None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:08.407098 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d40677-7452-48ed-a164-8a552697ca6c None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:08.407575 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d40677-7452-48ed-a164-8a552697ca6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:08.409365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d40677-7452-48ed-a164-8a552697ca6c None admin] RBAC: Adding query filter params (role.id=bdf71ef6ab10430e8c356cd4d952ff5c, scope.project.id=9f4373dfb5f9425eb73669e24d18a08f, user.id=cd7b11f479a54adc8d9b8641859b07a1) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:08.409672 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d40677-7452-48ed-a164-8a552697ca6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:08.411101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d40677-7452-48ed-a164-8a552697ca6c None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:08.417097 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 328/659] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:15:08 2022] GET /identity/v3/role_assignments?user.id=cd7b11f479a54adc8d9b8641859b07a1&scope.project.id=9f4373dfb5f9425eb73669e24d18a08f&role.id=bdf71ef6ab10430e8c356cd4d952ff5c => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:08.417698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d40677-7452-48ed-a164-8a552697ca6c None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:08.449660 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:08.466956 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ca4e2-8ba7-4886-bbd6-5a41fdcc36da None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:08.515813 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af13e6-fc32-43c0-9d5c-aa7661245823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:08.516511 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af13e6-fc32-43c0-9d5c-aa7661245823 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:08.516792 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af13e6-fc32-43c0-9d5c-aa7661245823 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:08.517065 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af13e6-fc32-43c0-9d5c-aa7661245823 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:08.543625 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d40677-7452-48ed-a164-8a552697ca6c None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:08.580841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:08.593206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf74e35a-4a82-4dac-9a13-6d35d2267caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:08.593905 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74e35a-4a82-4dac-9a13-6d35d2267caa None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:08.594184 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74e35a-4a82-4dac-9a13-6d35d2267caa None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:08.594454 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74e35a-4a82-4dac-9a13-6d35d2267caa None admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:08.595191 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf74e35a-4a82-4dac-9a13-6d35d2267caa None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:08.595491 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf74e35a-4a82-4dac-9a13-6d35d2267caa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:08.597688 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf74e35a-4a82-4dac-9a13-6d35d2267caa None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:08.598982 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af13e6-fc32-43c0-9d5c-aa7661245823 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:08.601683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af13e6-fc32-43c0-9d5c-aa7661245823 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:08.616839 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 329/660] 10.222.0.18 () {56 vars in 1137 bytes} [Tue Aug 23 04:15:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:08.618074 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf74e35a-4a82-4dac-9a13-6d35d2267caa None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:08.710463 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:08.719333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 332/661] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:15:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:15:08.720266 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af13e6-fc32-43c0-9d5c-aa7661245823 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:08.742143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf74e35a-4a82-4dac-9a13-6d35d2267caa None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:08.778515 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c9c94-7ffa-4e24-ba5b-3c37b83604c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:08.779128 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c9c94-7ffa-4e24-ba5b-3c37b83604c5 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:08.779403 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c9c94-7ffa-4e24-ba5b-3c37b83604c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:08.779685 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c9c94-7ffa-4e24-ba5b-3c37b83604c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:08.822766 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af13e6-fc32-43c0-9d5c-aa7661245823 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:08.831803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:08.852603 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988c9c94-7ffa-4e24-ba5b-3c37b83604c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:08.854082 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988c9c94-7ffa-4e24-ba5b-3c37b83604c5 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:08.862253 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18f745-7ae0-4bc7-8cf6-86ddc0398e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:08.862894 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18f745-7ae0-4bc7-8cf6-86ddc0398e12 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:08.863168 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18f745-7ae0-4bc7-8cf6-86ddc0398e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:08.863437 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18f745-7ae0-4bc7-8cf6-86ddc0398e12 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:08.874877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e18f745-7ae0-4bc7-8cf6-86ddc0398e12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:15:08.879837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-3e18f745-7ae0-4bc7-8cf6-86ddc0398e12 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:15:08.879837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:08.879837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:08.879837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:08.879837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:08.879837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:08.879837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:08.879837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:08.879837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:08.879837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:08.879837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:08.879837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:08.879837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:15:08.879837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:15:08.879837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:15:08.879837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:15:08.880750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:15:08.880750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:15:08.880750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:15:08.880750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:15:08.880750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:08.880750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:08.880750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:08.880750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:08.880750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:08.880750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:08.880750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:08.880750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:08.880750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:08.880750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:08.880750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:08.880750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:08.881887 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:08.881887 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:08.881887 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:08.881887 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:08.881887 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:15:08.881887 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:15:08.881887 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:15:08.881887 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:15:08.881887 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:15:08.881887 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:15:08.881887 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:15:08.881887 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:08.881887 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 333/662] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Aug 23 04:15:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:08.882579 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18f745-7ae0-4bc7-8cf6-86ddc0398e12 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:08.951381 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:08.967689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 330/663] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:15:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:15:08.969160 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c9c94-7ffa-4e24-ba5b-3c37b83604c5 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:08.998974 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18f745-7ae0-4bc7-8cf6-86ddc0398e12 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:09.036322 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5140088-c256-44a7-bd5b-6ccf60774af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:09.036927 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5140088-c256-44a7-bd5b-6ccf60774af7 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:09.037267 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5140088-c256-44a7-bd5b-6ccf60774af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:09.037564 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5140088-c256-44a7-bd5b-6ccf60774af7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:09.052893 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:09.071192 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c9c94-7ffa-4e24-ba5b-3c37b83604c5 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:09.105428 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0353d5dd-60b5-44ff-9700-8a724e845a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:09.106150 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353d5dd-60b5-44ff-9700-8a724e845a1c None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:09.106493 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353d5dd-60b5-44ff-9700-8a724e845a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:09.106832 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353d5dd-60b5-44ff-9700-8a724e845a1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:09.110429 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5140088-c256-44a7-bd5b-6ccf60774af7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:09.112120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5140088-c256-44a7-bd5b-6ccf60774af7 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:09.117587 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0353d5dd-60b5-44ff-9700-8a724e845a1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:09.119026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0353d5dd-60b5-44ff-9700-8a724e845a1c None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:09.133621 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-0353d5dd-60b5-44ff-9700-8a724e845a1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:09.133621 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:09.133621 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:09.133621 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:09.133621 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:09.133621 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:09.133621 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:09.133621 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:09.133621 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:09.133621 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:09.133621 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:09.133621 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:09.133621 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:09.133621 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:09.133621 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:09.133621 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:09.134552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:09.134552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:09.134552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:09.134552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:09.134552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:09.134552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:09.134552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:09.134552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:09.134552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:09.134552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:09.134552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:09.134552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:09.134552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:09.134552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:09.134552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:09.134552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:09.134552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:09.135420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:09.135420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:09.135420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:09.135420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 331/664] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:09.135420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0353d5dd-60b5-44ff-9700-8a724e845a1c None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:09.176247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:09.231662 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 334/665] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:15:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:15:09.233343 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5140088-c256-44a7-bd5b-6ccf60774af7 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:09.237639 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0353d5dd-60b5-44ff-9700-8a724e845a1c None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:09.279561 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496f0d6-f11a-438b-a620-a2fef07d5d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:09.280231 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496f0d6-f11a-438b-a620-a2fef07d5d85 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:09.280711 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496f0d6-f11a-438b-a620-a2fef07d5d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:09.281185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496f0d6-f11a-438b-a620-a2fef07d5d85 None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:09.282409 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496f0d6-f11a-438b-a620-a2fef07d5d85 None admin] RBAC: Adding query filter params (name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:09.282949 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496f0d6-f11a-438b-a620-a2fef07d5d85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:09.285196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496f0d6-f11a-438b-a620-a2fef07d5d85 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:09.292823 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:09.300569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 332/666] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Aug 23 04:15:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:09.301398 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496f0d6-f11a-438b-a620-a2fef07d5d85 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:09.353055 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5140088-c256-44a7-bd5b-6ccf60774af7 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:09.392604 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c066f93-e185-41ba-bc2d-55e2e5b90066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:09.393148 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c066f93-e185-41ba-bc2d-55e2e5b90066 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:09.393419 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c066f93-e185-41ba-bc2d-55e2e5b90066 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:09.393688 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c066f93-e185-41ba-bc2d-55e2e5b90066 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:09.406510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c066f93-e185-41ba-bc2d-55e2e5b90066 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:09.406697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496f0d6-f11a-438b-a620-a2fef07d5d85 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:09.408034 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c066f93-e185-41ba-bc2d-55e2e5b90066 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:09.424593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-8c066f93-e185-41ba-bc2d-55e2e5b90066 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:09.424593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:09.424593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:09.424593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:09.424593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:09.424593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:09.424593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:09.424593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:09.424593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:09.424593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:09.424593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:09.424593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:09.424593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:09.424593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:09.424593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:09.424593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:09.425520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:09.425520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:09.425520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:09.425520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:09.425520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:09.425520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:09.425520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:09.425520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:09.425520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:09.425520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:09.425520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:09.425520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:09.425520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:09.425520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:09.425520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:09.425520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:09.425520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:09.426441 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:09.426441 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:09.426441 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:09.428650 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 335/667] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:09.429737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c066f93-e185-41ba-bc2d-55e2e5b90066 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:09.440345 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b10f3-90b0-42e2-b2a7-067644bdd97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:09.440958 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b10f3-90b0-42e2-b2a7-067644bdd97f None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:09.441233 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b10f3-90b0-42e2-b2a7-067644bdd97f None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:09.441498 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b10f3-90b0-42e2-b2a7-067644bdd97f None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:09.442142 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b10f3-90b0-42e2-b2a7-067644bdd97f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:09.442441 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b10f3-90b0-42e2-b2a7-067644bdd97f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:09.443841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b10f3-90b0-42e2-b2a7-067644bdd97f None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:09.447220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:09.460198 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 333/668] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:09.460198 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b10f3-90b0-42e2-b2a7-067644bdd97f None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:09.527814 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:09.556217 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c066f93-e185-41ba-bc2d-55e2e5b90066 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:09.572041 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b10f3-90b0-42e2-b2a7-067644bdd97f None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:09.596290 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75651622-63bc-4123-ae7b-af17b3869ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:09.596929 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75651622-63bc-4123-ae7b-af17b3869ccb None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:09.597201 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75651622-63bc-4123-ae7b-af17b3869ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:09.597469 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75651622-63bc-4123-ae7b-af17b3869ccb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:09.608785 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8335790c-597c-438c-afac-8c0154c49542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:09.609440 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335790c-597c-438c-afac-8c0154c49542 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:09.609718 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335790c-597c-438c-afac-8c0154c49542 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:09.609842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75651622-63bc-4123-ae7b-af17b3869ccb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:09.610008 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335790c-597c-438c-afac-8c0154c49542 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:09.611296 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75651622-63bc-4123-ae7b-af17b3869ccb None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:09.620944 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8335790c-597c-438c-afac-8c0154c49542 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:09.622400 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8335790c-597c-438c-afac-8c0154c49542 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:09.626576 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-75651622-63bc-4123-ae7b-af17b3869ccb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:09.626576 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:09.626576 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:09.626576 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:09.626576 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:09.626576 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:09.626576 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:09.626576 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:09.626576 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:09.626576 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:09.626576 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:09.626576 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:09.626576 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:09.626576 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:09.626576 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:09.626576 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:09.627492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:09.627492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:09.627492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:09.627492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:09.627492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:09.627492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:09.627492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:09.627492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:09.627492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:09.627492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:09.627492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:09.627492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:09.627492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:09.627492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:09.627492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:09.627492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:09.627492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:09.628419 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:09.628419 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:09.628419 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:09.628419 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 336/669] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:09.629145 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75651622-63bc-4123-ae7b-af17b3869ccb None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:09.642079 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-8335790c-597c-438c-afac-8c0154c49542 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:09.642079 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:09.642079 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:09.642079 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:09.642079 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:09.642079 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:09.642079 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:09.642079 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:09.642079 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:09.642079 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:09.642079 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:09.642079 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:09.642079 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:09.642079 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:09.642079 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:09.642079 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:09.643154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:09.643154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:09.643154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:09.643154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:09.643154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:09.643154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:09.643154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:09.643154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:09.643154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:09.643154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:09.643154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:09.643154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:09.643154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:09.643154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:09.643154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:09.643154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:09.643154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:09.644081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:09.644081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:09.644081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:09.644081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 334/670] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:09.644081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8335790c-597c-438c-afac-8c0154c49542 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:09.696911 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:09.743937 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75651622-63bc-4123-ae7b-af17b3869ccb None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:09.750315 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8335790c-597c-438c-afac-8c0154c49542 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:09.762181 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:09.779601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493ce6b-7760-4191-ba51-5501949364d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:09.780247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493ce6b-7760-4191-ba51-5501949364d9 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:09.780526 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493ce6b-7760-4191-ba51-5501949364d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:09.780802 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493ce6b-7760-4191-ba51-5501949364d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:09.781447 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2493ce6b-7760-4191-ba51-5501949364d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:09.781745 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2493ce6b-7760-4191-ba51-5501949364d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:09.783118 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2493ce6b-7760-4191-ba51-5501949364d9 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:09.789359 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05db5e1-e9e8-4934-8bb0-c81a8bfd3865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:09.790197 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05db5e1-e9e8-4934-8bb0-c81a8bfd3865 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:09.791112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05db5e1-e9e8-4934-8bb0-c81a8bfd3865 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:09.791586 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05db5e1-e9e8-4934-8bb0-c81a8bfd3865 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:09.795971 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 337/671] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:09.796782 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493ce6b-7760-4191-ba51-5501949364d9 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:09.799549 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05db5e1-e9e8-4934-8bb0-c81a8bfd3865 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:09.801045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05db5e1-e9e8-4934-8bb0-c81a8bfd3865 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:09.815346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-b05db5e1-e9e8-4934-8bb0-c81a8bfd3865 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:09.815346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:09.815346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:09.815346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:09.815346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:09.815346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:09.815346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:09.815346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:09.815346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:09.815346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:09.815346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:09.815346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:09.815346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:15:09.815346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:15:09.815346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:15:09.816588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:15:09.816588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:09.816588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:09.816588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:09.816588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:09.816588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:09.816588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:09.816588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:09.816588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:09.816588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:09.816588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:09.816588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:09.816588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:09.816588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:09.816588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:09.816588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:09.816588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:09.822520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:15:09.822520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:15:09.822520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:15:09.822520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:15:09.822520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:15:09.822520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:15:09.822520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:09.822520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:09.822520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 335/672] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:15:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:09.822520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05db5e1-e9e8-4934-8bb0-c81a8bfd3865 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:09.909571 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493ce6b-7760-4191-ba51-5501949364d9 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:09.934029 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05db5e1-e9e8-4934-8bb0-c81a8bfd3865 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:09.934296 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:09.952003 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa995a3f-48f8-4809-9df2-ead94665bed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:09.952655 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa995a3f-48f8-4809-9df2-ead94665bed0 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:09.952887 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa995a3f-48f8-4809-9df2-ead94665bed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:09.953152 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa995a3f-48f8-4809-9df2-ead94665bed0 None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:09.953792 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa995a3f-48f8-4809-9df2-ead94665bed0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:09.954090 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa995a3f-48f8-4809-9df2-ead94665bed0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:09.955449 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa995a3f-48f8-4809-9df2-ead94665bed0 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:09.972904 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 338/673] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:09.972904 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa995a3f-48f8-4809-9df2-ead94665bed0 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:09.976882 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110dc493-f996-45a8-800a-0a080e5effee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:09.978378 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110dc493-f996-45a8-800a-0a080e5effee None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:09.978662 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110dc493-f996-45a8-800a-0a080e5effee None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:09.978933 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110dc493-f996-45a8-800a-0a080e5effee None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:09.979585 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110dc493-f996-45a8-800a-0a080e5effee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:09.979889 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110dc493-f996-45a8-800a-0a080e5effee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:09.981301 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110dc493-f996-45a8-800a-0a080e5effee None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:09.996219 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 336/674] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:09.997071 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110dc493-f996-45a8-800a-0a080e5effee None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:10.029792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:10.089350 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa995a3f-48f8-4809-9df2-ead94665bed0 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:10.104999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110dc493-f996-45a8-800a-0a080e5effee None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:10.122340 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa48b8-f92a-41e2-b543-b0709d229f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:10.122979 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa48b8-f92a-41e2-b543-b0709d229f02 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:10.123253 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa48b8-f92a-41e2-b543-b0709d229f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:10.123520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa48b8-f92a-41e2-b543-b0709d229f02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:10.130020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afa48b8-f92a-41e2-b543-b0709d229f02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:10.131653 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afa48b8-f92a-41e2-b543-b0709d229f02 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:10.141456 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726d01e-9827-4006-8201-aa68bf9749ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:10.142065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726d01e-9827-4006-8201-aa68bf9749ae None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:10.142339 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726d01e-9827-4006-8201-aa68bf9749ae None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:10.142606 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726d01e-9827-4006-8201-aa68bf9749ae None admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:10.143293 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d726d01e-9827-4006-8201-aa68bf9749ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:10.143590 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d726d01e-9827-4006-8201-aa68bf9749ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:10.144594 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-9afa48b8-f92a-41e2-b543-b0709d229f02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:10.144594 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:10.144594 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:10.144594 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:10.144594 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:10.144594 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:10.144594 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:10.144594 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:10.144594 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:10.144594 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:10.144594 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:10.144594 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:10.144594 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:15:10.144594 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:15:10.144594 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:15:10.145567 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:15:10.145567 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:10.145567 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:10.145567 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:10.145567 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:10.145567 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:10.145567 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:10.145567 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:10.145567 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:10.145567 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:10.145567 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:10.145567 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:10.145567 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:10.145567 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:10.145567 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:10.145567 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:10.145567 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:10.146682 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:15:10.146682 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:15:10.146682 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:15:10.146682 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:15:10.146682 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:15:10.146682 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:15:10.146682 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:10.146682 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:10.147187 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d726d01e-9827-4006-8201-aa68bf9749ae None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:10.148129 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 339/675] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:15:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:10.153283 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa48b8-f92a-41e2-b543-b0709d229f02 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:10.158468 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 337/676] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:15:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:10.159477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726d01e-9827-4006-8201-aa68bf9749ae None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:10.200140 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:10.260054 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa48b8-f92a-41e2-b543-b0709d229f02 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:10.264147 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:10.273685 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726d01e-9827-4006-8201-aa68bf9749ae None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:10.296077 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc61d11-d5c2-4e66-989e-065d443c126d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:10.296894 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc61d11-d5c2-4e66-989e-065d443c126d None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:10.297077 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc61d11-d5c2-4e66-989e-065d443c126d None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:10.297355 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc61d11-d5c2-4e66-989e-065d443c126d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:10.304644 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc61d11-d5c2-4e66-989e-065d443c126d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:10.306096 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc61d11-d5c2-4e66-989e-065d443c126d None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:10.309721 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afd35e-f51a-4303-9efa-7c9f0fbf11bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:10.310429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afd35e-f51a-4303-9efa-7c9f0fbf11bb None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:10.310762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afd35e-f51a-4303-9efa-7c9f0fbf11bb None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:10.311090 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afd35e-f51a-4303-9efa-7c9f0fbf11bb None admin] PATH_INFO: `/v3/users/swift` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:10.317476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-0cc61d11-d5c2-4e66-989e-065d443c126d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:10.317476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:10.317476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:10.317476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:10.317476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:10.317476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:10.317476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:10.317476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:10.317476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:10.317476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:10.317476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:10.317476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:10.317476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:15:10.317476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:15:10.317476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:15:10.318577 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:15:10.318577 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:10.318577 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:10.318577 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:10.318577 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:10.318577 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:10.318577 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:10.318577 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:10.318577 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:10.318577 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:10.318577 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:10.318577 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:10.318577 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:10.318577 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:10.318577 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:10.318577 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:10.318577 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:10.319508 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:15:10.319508 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:15:10.319508 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:15:10.319508 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:15:10.319508 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:15:10.319508 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:15:10.319508 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:10.319508 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:10.319508 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 340/677] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:15:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:10.320026 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc61d11-d5c2-4e66-989e-065d443c126d None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:10.325185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1afd35e-f51a-4303-9efa-7c9f0fbf11bb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:10.326652 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1afd35e-f51a-4303-9efa-7c9f0fbf11bb None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:10.338895 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-a1afd35e-f51a-4303-9efa-7c9f0fbf11bb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 04:15:10.338895 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:10.338895 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:10.338895 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:10.338895 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:10.338895 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:10.338895 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:10.338895 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:10.338895 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:10.338895 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:10.338895 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:10.338895 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:10.338895 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:15:10.338895 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:15:10.338895 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:15:10.338895 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:15:10.339826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:10.339826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:10.339826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:15:10.339826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:10.339826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:15:10.339826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:10.339826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:10.339826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:10.339826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:10.339826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:10.339826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:10.339826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:10.339826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:10.339826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:10.339826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:10.339826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:10.339826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:10.339826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:10.340783 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:10.340783 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:10.340783 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:15:10.340783 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:15:10.340783 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:15:10.340783 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:15:10.340783 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:15:10.340783 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:15:10.340783 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 04:15:10.340783 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:10.340783 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 338/678] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:15:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:10.340783 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afd35e-f51a-4303-9efa-7c9f0fbf11bb None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:10.428015 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:10.431211 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc61d11-d5c2-4e66-989e-065d443c126d None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:10.447302 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afd35e-f51a-4303-9efa-7c9f0fbf11bb None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:10.465534 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4b9aa-f01c-4988-a10a-50c590dfc108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:10.466139 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4b9aa-f01c-4988-a10a-50c590dfc108 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:10.466442 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4b9aa-f01c-4988-a10a-50c590dfc108 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:10.466711 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4b9aa-f01c-4988-a10a-50c590dfc108 None admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:10.467397 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc4b9aa-f01c-4988-a10a-50c590dfc108 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:10.467695 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc4b9aa-f01c-4988-a10a-50c590dfc108 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:10.469162 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc4b9aa-f01c-4988-a10a-50c590dfc108 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:10.481360 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 341/679] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:15:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:10.481932 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4b9aa-f01c-4988-a10a-50c590dfc108 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:10.488221 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4203f-d260-49fd-8101-2e63e897380f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:10.488834 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4203f-d260-49fd-8101-2e63e897380f None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:10.489111 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4203f-d260-49fd-8101-2e63e897380f None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:10.489379 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4203f-d260-49fd-8101-2e63e897380f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:10.490191 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4203f-d260-49fd-8101-2e63e897380f None admin] RBAC: Adding query filter params (role.id=bdf71ef6ab10430e8c356cd4d952ff5c, scope.project.id=9f4373dfb5f9425eb73669e24d18a08f, user.id=4f1bf93cff6c46e0801bb827203cb5ad) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:10.490509 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4203f-d260-49fd-8101-2e63e897380f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:10.492008 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4203f-d260-49fd-8101-2e63e897380f None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:10.498265 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 339/680] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:15:10 2022] GET /identity/v3/role_assignments?user.id=4f1bf93cff6c46e0801bb827203cb5ad&scope.project.id=9f4373dfb5f9425eb73669e24d18a08f&role.id=bdf71ef6ab10430e8c356cd4d952ff5c => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:10.499032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4203f-d260-49fd-8101-2e63e897380f None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:10.530847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:10.597785 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4b9aa-f01c-4988-a10a-50c590dfc108 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:10.633046 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4203f-d260-49fd-8101-2e63e897380f None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:10.637701 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229f9aa-b3f1-4d6d-aa04-e2376d194ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:10.638327 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229f9aa-b3f1-4d6d-aa04-e2376d194ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:10.638603 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229f9aa-b3f1-4d6d-aa04-e2376d194ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:10.638874 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229f9aa-b3f1-4d6d-aa04-e2376d194ee1 None admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:10.639575 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3229f9aa-b3f1-4d6d-aa04-e2376d194ee1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:10.639883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3229f9aa-b3f1-4d6d-aa04-e2376d194ee1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:10.641354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3229f9aa-b3f1-4d6d-aa04-e2376d194ee1 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:10.653939 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 342/681] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:15:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:10.655127 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792c10f-6d5c-4ca1-920d-837bd520ade4 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:10.655410 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792c10f-6d5c-4ca1-920d-837bd520ade4 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:10.655677 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792c10f-6d5c-4ca1-920d-837bd520ade4 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:10.660968 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 343/682] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:10.660968 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792c10f-6d5c-4ca1-920d-837bd520ade4 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:10.676510 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43ee013-de3b-45f7-a5e9-548476dbef9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:10.677097 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ee013-de3b-45f7-a5e9-548476dbef9f None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:10.677375 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ee013-de3b-45f7-a5e9-548476dbef9f None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:10.677763 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ee013-de3b-45f7-a5e9-548476dbef9f None admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:10.679167 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43ee013-de3b-45f7-a5e9-548476dbef9f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:10.679473 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43ee013-de3b-45f7-a5e9-548476dbef9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:10.680951 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43ee013-de3b-45f7-a5e9-548476dbef9f None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:10.684437 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:10.701932 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 340/683] 10.222.0.18 () {56 vars in 1135 bytes} [Tue Aug 23 04:15:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:10.702782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43ee013-de3b-45f7-a5e9-548476dbef9f None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:10.775799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792c10f-6d5c-4ca1-920d-837bd520ade4 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:10.780943 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:10.812979 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43ee013-de3b-45f7-a5e9-548476dbef9f None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:10.813931 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08697d8d-5a34-4e51-bb18-c178396ec478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:10.814546 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08697d8d-5a34-4e51-bb18-c178396ec478 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:10.814822 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08697d8d-5a34-4e51-bb18-c178396ec478 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:10.815094 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08697d8d-5a34-4e51-bb18-c178396ec478 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:10.815849 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08697d8d-5a34-4e51-bb18-c178396ec478 None admin] RBAC: Adding query filter params (role.id=bdf71ef6ab10430e8c356cd4d952ff5c, scope.project.id=9f4373dfb5f9425eb73669e24d18a08f, user.id=fdca9c48e5d740f6bda8c48072b54c0b) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:10.816172 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08697d8d-5a34-4e51-bb18-c178396ec478 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:10.817603 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08697d8d-5a34-4e51-bb18-c178396ec478 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:10.824836 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 344/684] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:15:10 2022] GET /identity/v3/role_assignments?user.id=fdca9c48e5d740f6bda8c48072b54c0b&scope.project.id=9f4373dfb5f9425eb73669e24d18a08f&role.id=bdf71ef6ab10430e8c356cd4d952ff5c => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:10.833549 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3217c-a467-413b-bc89-4605454ba6b5 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:10.833943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3217c-a467-413b-bc89-4605454ba6b5 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:10.834170 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3217c-a467-413b-bc89-4605454ba6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:10.855220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-f2b3217c-a467-413b-bc89-4605454ba6b5 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:10.856560 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-f2b3217c-a467-413b-bc89-4605454ba6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:10.861199 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc378c-b3ec-4934-bec0-881b6c230cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:10.862250 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc378c-b3ec-4934-bec0-881b6c230cba None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:10.862751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc378c-b3ec-4934-bec0-881b6c230cba None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:10.863222 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc378c-b3ec-4934-bec0-881b6c230cba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:10.864221 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbc378c-b3ec-4934-bec0-881b6c230cba None admin] RBAC: Adding query filter params (role.id=bdf71ef6ab10430e8c356cd4d952ff5c, scope.project.id=9f4373dfb5f9425eb73669e24d18a08f, user.id=2db57fe3915446acb57b4f84ca353379) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:10.864844 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbc378c-b3ec-4934-bec0-881b6c230cba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:10.867423 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbc378c-b3ec-4934-bec0-881b6c230cba None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:10.876503 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 341/685] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:15:10 2022] GET /identity/v3/role_assignments?user.id=2db57fe3915446acb57b4f84ca353379&scope.project.id=9f4373dfb5f9425eb73669e24d18a08f&role.id=bdf71ef6ab10430e8c356cd4d952ff5c => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:10.877755 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc378c-b3ec-4934-bec0-881b6c230cba None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:11.012316 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc378c-b3ec-4934-bec0-881b6c230cba None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:11.028035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 345/686] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:15:11.058547 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc1ca7-efe0-4f2a-b458-ddc6493cd0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:11.059371 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc1ca7-efe0-4f2a-b458-ddc6493cd0ec None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:11.059754 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc1ca7-efe0-4f2a-b458-ddc6493cd0ec None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:11.060146 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc1ca7-efe0-4f2a-b458-ddc6493cd0ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:11.065306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:11.138761 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc1ca7-efe0-4f2a-b458-ddc6493cd0ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:11.140413 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc1ca7-efe0-4f2a-b458-ddc6493cd0ec None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:11.246127 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 342/687] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:15:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:15:11.251634 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3217c-a467-413b-bc89-4605454ba6b5 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:11.270047 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:11.338955 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc1ca7-efe0-4f2a-b458-ddc6493cd0ec None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:11.369733 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3217c-a467-413b-bc89-4605454ba6b5 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:11.378230 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:11.419257 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada627e-5376-45d9-af7f-c4ec5913b62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:11.420429 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada627e-5376-45d9-af7f-c4ec5913b62c None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:11.424267 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada627e-5376-45d9-af7f-c4ec5913b62c None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:11.424541 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada627e-5376-45d9-af7f-c4ec5913b62c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:11.441326 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada627e-5376-45d9-af7f-c4ec5913b62c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:11.444324 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada627e-5376-45d9-af7f-c4ec5913b62c None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:11.454673 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc1ca7-efe0-4f2a-b458-ddc6493cd0ec None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:11.468338 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-5ada627e-5376-45d9-af7f-c4ec5913b62c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:11.468338 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:11.468338 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:11.468338 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:11.468338 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:11.468338 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:11.468338 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:11.468338 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:11.468338 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:11.468338 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:11.468338 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:11.468338 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:11.468338 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:11.468338 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:11.468338 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:11.468338 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:11.469278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:11.469278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:11.469278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:11.469278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:11.469278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:11.469278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:11.469278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:11.469278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:11.469278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:11.469278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:11.469278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:11.469278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:11.469278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:11.469278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:11.469278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:11.469278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:11.469278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:11.470153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:11.470153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:11.470153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:11.470749 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 346/688] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:11.476600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada627e-5376-45d9-af7f-c4ec5913b62c None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:11.513490 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e824c-fe1f-45d5-855c-5b4a91438f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:11.514139 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e824c-fe1f-45d5-855c-5b4a91438f0b None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:11.514416 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e824c-fe1f-45d5-855c-5b4a91438f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:11.514683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e824c-fe1f-45d5-855c-5b4a91438f0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:11.532490 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e824c-fe1f-45d5-855c-5b4a91438f0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:15:11.537182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-230e824c-fe1f-45d5-855c-5b4a91438f0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:15:11.537182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:11.537182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:11.537182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:11.537182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:11.537182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:11.537182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:11.537182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:11.537182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:11.537182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:11.537182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:11.537182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:11.537182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:15:11.537182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:15:11.537182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:15:11.537182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:15:11.539869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:15:11.539869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:15:11.539869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:15:11.539869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:15:11.539869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:11.539869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:11.539869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:11.539869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:11.539869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:11.539869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:11.539869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:11.539869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:11.539869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:11.539869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:11.539869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:11.539869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:11.541849 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:11.541849 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:11.541849 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:11.541849 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:11.541849 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:15:11.541849 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:15:11.541849 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:15:11.541849 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:15:11.541849 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:15:11.541849 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:15:11.541849 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:15:11.541849 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:11.541849 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 343/689] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:15:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:15:11.547687 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e824c-fe1f-45d5-855c-5b4a91438f0b None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:11.620803 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:11.661035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada627e-5376-45d9-af7f-c4ec5913b62c None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:11.713973 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:11.720259 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e824c-fe1f-45d5-855c-5b4a91438f0b None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:11.753155 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59993c1-e1c3-4975-b0d2-7fb625b6476f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:11.753774 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59993c1-e1c3-4975-b0d2-7fb625b6476f None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:11.754053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59993c1-e1c3-4975-b0d2-7fb625b6476f None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:11.754319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59993c1-e1c3-4975-b0d2-7fb625b6476f None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:11.754962 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59993c1-e1c3-4975-b0d2-7fb625b6476f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:11.755256 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59993c1-e1c3-4975-b0d2-7fb625b6476f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:11.757156 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59993c1-e1c3-4975-b0d2-7fb625b6476f None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:11.765592 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42082d32-d583-4973-905d-b0654e81fad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:11.766219 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42082d32-d583-4973-905d-b0654e81fad3 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:11.766645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42082d32-d583-4973-905d-b0654e81fad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:11.766887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42082d32-d583-4973-905d-b0654e81fad3 None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:11.767557 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42082d32-d583-4973-905d-b0654e81fad3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:11.767855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42082d32-d583-4973-905d-b0654e81fad3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:11.769252 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42082d32-d583-4973-905d-b0654e81fad3 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:11.771016 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 347/690] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:11.776926 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59993c1-e1c3-4975-b0d2-7fb625b6476f None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:11.778566 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 344/691] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:15:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:11.783805 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42082d32-d583-4973-905d-b0654e81fad3 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:11.941204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42082d32-d583-4973-905d-b0654e81fad3 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:11.978426 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59993c1-e1c3-4975-b0d2-7fb625b6476f None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:12.014598 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:12.024271 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9392889b-91bd-470b-808d-fa47a7a9f824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:12.024623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905eb55a-1468-4974-8f60-8eb5fa35e742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:12.025093 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392889b-91bd-470b-808d-fa47a7a9f824 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:12.025174 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392889b-91bd-470b-808d-fa47a7a9f824 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:12.025364 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905eb55a-1468-4974-8f60-8eb5fa35e742 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:12.025458 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392889b-91bd-470b-808d-fa47a7a9f824 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:12.025598 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905eb55a-1468-4974-8f60-8eb5fa35e742 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:12.025849 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905eb55a-1468-4974-8f60-8eb5fa35e742 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:12.032720 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9392889b-91bd-470b-808d-fa47a7a9f824 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:12.034152 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9392889b-91bd-470b-808d-fa47a7a9f824 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:12.046660 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-9392889b-91bd-470b-808d-fa47a7a9f824 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:12.046660 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:12.046660 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:12.046660 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:12.046660 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:12.046660 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:12.046660 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:12.046660 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:12.046660 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:12.046660 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:12.046660 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:12.046660 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:12.046660 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:15:12.046660 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:15:12.046660 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:15:12.047572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:15:12.047572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:12.047572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:12.047572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:12.047572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:12.047572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:12.047572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:12.047572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:12.047572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:12.047572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:12.047572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:12.047572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:12.047572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:12.047572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:12.047572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:12.047572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:12.047572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:12.048473 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:15:12.048473 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:15:12.048473 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:15:12.048473 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:15:12.048473 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:15:12.048473 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:15:12.048473 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:12.048473 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:12.052271 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 348/692] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:15:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:12.053890 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905eb55a-1468-4974-8f60-8eb5fa35e742 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:12.054097 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9392889b-91bd-470b-808d-fa47a7a9f824 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:12.056720 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905eb55a-1468-4974-8f60-8eb5fa35e742 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:12.091556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-905eb55a-1468-4974-8f60-8eb5fa35e742 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:12.091556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:12.091556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:12.091556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:12.091556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:12.091556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:12.091556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:12.091556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:12.091556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:12.091556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:12.091556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:12.091556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:12.091556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:12.091556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:12.091556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:12.091556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:12.093765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:12.093765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:12.093765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:12.093765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:12.093765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:12.093765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:12.093765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:12.093765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:12.093765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:12.093765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:12.093765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:12.093765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:12.093765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:12.093765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:12.093765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:12.093765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:12.093765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:12.095797 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:12.095797 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:12.095797 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:12.095797 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 345/693] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:12.100874 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905eb55a-1468-4974-8f60-8eb5fa35e742 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:12.132266 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:12.206009 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9392889b-91bd-470b-808d-fa47a7a9f824 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:12.241187 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf838134-49f4-41c6-9275-88e8e8ba08bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:12.241800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf838134-49f4-41c6-9275-88e8e8ba08bb None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:12.242073 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf838134-49f4-41c6-9275-88e8e8ba08bb None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:12.242486 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf838134-49f4-41c6-9275-88e8e8ba08bb None admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:12.243176 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf838134-49f4-41c6-9275-88e8e8ba08bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:12.243474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf838134-49f4-41c6-9275-88e8e8ba08bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:12.244932 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf838134-49f4-41c6-9275-88e8e8ba08bb None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:12.258098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 349/694] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:15:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:12.264167 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf838134-49f4-41c6-9275-88e8e8ba08bb None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:12.316372 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:12.373327 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf838134-49f4-41c6-9275-88e8e8ba08bb None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:12.414810 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc0d1b5-809d-46e9-95ef-feb7e3f83ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:12.415429 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc0d1b5-809d-46e9-95ef-feb7e3f83ef1 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:12.415709 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc0d1b5-809d-46e9-95ef-feb7e3f83ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:12.416006 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc0d1b5-809d-46e9-95ef-feb7e3f83ef1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:12.416738 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc0d1b5-809d-46e9-95ef-feb7e3f83ef1 None admin] RBAC: Adding query filter params (role.id=bdf71ef6ab10430e8c356cd4d952ff5c, scope.project.id=9f4373dfb5f9425eb73669e24d18a08f, user.id=3f1418fd211a448da1a50175dce13b32) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:12.417044 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc0d1b5-809d-46e9-95ef-feb7e3f83ef1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:12.418590 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc0d1b5-809d-46e9-95ef-feb7e3f83ef1 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:12.425892 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 350/695] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:15:12 2022] GET /identity/v3/role_assignments?user.id=3f1418fd211a448da1a50175dce13b32&scope.project.id=9f4373dfb5f9425eb73669e24d18a08f&role.id=bdf71ef6ab10430e8c356cd4d952ff5c => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:12.468583 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905eb55a-1468-4974-8f60-8eb5fa35e742 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:12.539429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35c09e-5220-4d82-b315-e5fedeb637e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:12.540800 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35c09e-5220-4d82-b315-e5fedeb637e8 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:12.541265 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35c09e-5220-4d82-b315-e5fedeb637e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:12.541713 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35c09e-5220-4d82-b315-e5fedeb637e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:12.542647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c35c09e-5220-4d82-b315-e5fedeb637e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:12.543331 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c35c09e-5220-4d82-b315-e5fedeb637e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:12.545117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c35c09e-5220-4d82-b315-e5fedeb637e8 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:12.560656 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 346/696] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 460 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:12.567416 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc0d1b5-809d-46e9-95ef-feb7e3f83ef1 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:12.712064 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc0d1b5-809d-46e9-95ef-feb7e3f83ef1 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:12.726432 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:12.755424 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb9514-b88c-4c49-9631-19191ba46676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:12.756269 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb9514-b88c-4c49-9631-19191ba46676 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:12.756785 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb9514-b88c-4c49-9631-19191ba46676 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:12.756785 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb9514-b88c-4c49-9631-19191ba46676 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:12.765899 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bb9514-b88c-4c49-9631-19191ba46676 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:12.767368 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bb9514-b88c-4c49-9631-19191ba46676 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:12.778838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-74bb9514-b88c-4c49-9631-19191ba46676 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 04:15:12.778838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:12.778838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:12.778838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:12.778838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:12.778838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:12.778838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:12.778838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:12.778838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:12.778838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:12.778838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:12.778838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:12.778838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:15:12.778838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:15:12.778838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:15:12.778838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:15:12.779772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:12.779772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:12.779772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:15:12.779772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:12.779772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:15:12.779772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:12.779772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:12.779772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:12.779772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:12.779772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:12.779772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:12.779772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:12.779772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:12.779772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:12.779772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:12.779772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:12.779772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:12.779772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:12.780735 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:12.780735 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:12.780735 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:15:12.780735 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:15:12.780735 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:15:12.780735 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:15:12.780735 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:15:12.780735 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:15:12.780735 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 04:15:12.780735 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:12.784048 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 351/697] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Aug 23 04:15:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:12.786188 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35c09e-5220-4d82-b315-e5fedeb637e8 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:12.847335 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3c465-eb60-47f0-a547-06a69321bc62 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:12.849732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3c465-eb60-47f0-a547-06a69321bc62 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:12.849732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3c465-eb60-47f0-a547-06a69321bc62 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:12.849732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 352/698] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:15:12.854904 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb1ea3-2cfd-483f-b959-e68ec305b0aa None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:12.855210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb1ea3-2cfd-483f-b959-e68ec305b0aa None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:12.855595 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb1ea3-2cfd-483f-b959-e68ec305b0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:12.864832 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:12.875692 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-84cb1ea3-2cfd-483f-b959-e68ec305b0aa None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:12.876945 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-84cb1ea3-2cfd-483f-b959-e68ec305b0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:12.902739 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35c09e-5220-4d82-b315-e5fedeb637e8 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:12.947290 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74caef59-74dd-44d5-8554-e1b379928aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:12.947942 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74caef59-74dd-44d5-8554-e1b379928aea None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:12.948233 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74caef59-74dd-44d5-8554-e1b379928aea None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:12.948504 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74caef59-74dd-44d5-8554-e1b379928aea None admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:12.949255 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74caef59-74dd-44d5-8554-e1b379928aea None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:12.949560 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74caef59-74dd-44d5-8554-e1b379928aea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:12.951018 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74caef59-74dd-44d5-8554-e1b379928aea None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:12.954849 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 353/699] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 04:15:12.973458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 347/700] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Aug 23 04:15:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:12.979170 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb1ea3-2cfd-483f-b959-e68ec305b0aa None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:13.093924 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:13.097104 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb1ea3-2cfd-483f-b959-e68ec305b0aa None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:13.145554 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd5bc8-4ef6-4b17-b652-b50e3075c277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:13.146183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd5bc8-4ef6-4b17-b652-b50e3075c277 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:13.146523 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd5bc8-4ef6-4b17-b652-b50e3075c277 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:13.146827 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd5bc8-4ef6-4b17-b652-b50e3075c277 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:13.190910 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938dae3e-8df4-41ab-a690-a64efd9f9694 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:13.191440 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938dae3e-8df4-41ab-a690-a64efd9f9694 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:13.191440 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938dae3e-8df4-41ab-a690-a64efd9f9694 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:13.193716 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 348/701] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:13.201241 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130c35a-6730-48b9-ab11-240a8b6de882 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:13.201521 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130c35a-6730-48b9-ab11-240a8b6de882 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:13.201795 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130c35a-6730-48b9-ab11-240a8b6de882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:13.229168 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dd5bc8-4ef6-4b17-b652-b50e3075c277 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:13.231038 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dd5bc8-4ef6-4b17-b652-b50e3075c277 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:13.239443 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-1130c35a-6730-48b9-ab11-240a8b6de882 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:13.240495 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-1130c35a-6730-48b9-ab11-240a8b6de882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:13.330874 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:13.335787 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 354/702] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:15:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 04:15:13.336862 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd5bc8-4ef6-4b17-b652-b50e3075c277 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:13.345765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 349/703] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 23 04:15:13.347724 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb35415-af4d-47b0-94d3-b9a1561e6b81 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:13.348350 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb35415-af4d-47b0-94d3-b9a1561e6b81 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:13.348815 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb35415-af4d-47b0-94d3-b9a1561e6b81 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:13.352276 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 350/704] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:13.353535 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb35415-af4d-47b0-94d3-b9a1561e6b81 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:13.430564 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:13.452101 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd5bc8-4ef6-4b17-b652-b50e3075c277 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:13.483827 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb35415-af4d-47b0-94d3-b9a1561e6b81 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:13.491710 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da866269-af25-4385-a6a6-60ecdda5d88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:13.492788 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da866269-af25-4385-a6a6-60ecdda5d88c None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:13.493298 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da866269-af25-4385-a6a6-60ecdda5d88c None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:13.493776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da866269-af25-4385-a6a6-60ecdda5d88c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:13.505591 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da866269-af25-4385-a6a6-60ecdda5d88c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:13.507621 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da866269-af25-4385-a6a6-60ecdda5d88c None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:13.522074 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df551101-d363-4ef1-b483-9248a83b370f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:13.522769 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df551101-d363-4ef1-b483-9248a83b370f None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:13.523045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df551101-d363-4ef1-b483-9248a83b370f None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:13.523309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df551101-d363-4ef1-b483-9248a83b370f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:13.524841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-da866269-af25-4385-a6a6-60ecdda5d88c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:13.524841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:13.524841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:13.524841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:13.524841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:13.524841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:13.524841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:13.524841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:13.524841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:13.524841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:13.524841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:13.524841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:13.524841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:13.524841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:13.524841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:13.524841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:13.525777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:13.525777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:13.525777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:13.525777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:13.525777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:13.525777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:13.525777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:13.525777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:13.525777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:13.525777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:13.525777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:13.525777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:13.525777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:13.525777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:13.525777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:13.525777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:13.525777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:13.526721 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:13.526721 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:13.526721 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:13.529023 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 355/705] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:13.530650 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2e5b3-1499-410e-93c4-86b814db9bb6 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:13.531183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2e5b3-1499-410e-93c4-86b814db9bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:13.531687 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2e5b3-1499-410e-93c4-86b814db9bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:13.538870 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df551101-d363-4ef1-b483-9248a83b370f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:13.541153 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df551101-d363-4ef1-b483-9248a83b370f None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:13.551779 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-06a2e5b3-1499-410e-93c4-86b814db9bb6 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:13.553414 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-06a2e5b3-1499-410e-93c4-86b814db9bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:13.557812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-df551101-d363-4ef1-b483-9248a83b370f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:13.557812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:13.557812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:13.557812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:13.557812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:13.557812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:13.557812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:13.557812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:13.557812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:13.557812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:13.557812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:13.557812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:13.557812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:13.557812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:13.557812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:13.557812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:13.558751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:13.558751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:13.558751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:13.558751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:13.558751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:13.558751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:13.558751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:13.558751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:13.558751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:13.558751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:13.558751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:13.558751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:13.558751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:13.558751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:13.558751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:13.558751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:13.558751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:13.560279 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:13.560279 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:13.560279 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:13.560279 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 351/706] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:13.560279 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df551101-d363-4ef1-b483-9248a83b370f None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:13.612311 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:13.641414 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 356/707] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:15:13.642471 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2e5b3-1499-410e-93c4-86b814db9bb6 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:13.686532 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df551101-d363-4ef1-b483-9248a83b370f None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:13.700983 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:13.731876 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c02cb-3818-4db7-bfed-bb9e76f9f5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:13.733048 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c02cb-3818-4db7-bfed-bb9e76f9f5cc None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:13.733527 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c02cb-3818-4db7-bfed-bb9e76f9f5cc None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:13.734259 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c02cb-3818-4db7-bfed-bb9e76f9f5cc None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:13.735953 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449c02cb-3818-4db7-bfed-bb9e76f9f5cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:13.736436 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449c02cb-3818-4db7-bfed-bb9e76f9f5cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:13.737978 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449c02cb-3818-4db7-bfed-bb9e76f9f5cc None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:13.758565 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 352/708] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:13.760193 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c02cb-3818-4db7-bfed-bb9e76f9f5cc None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:13.776585 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2e5b3-1499-410e-93c4-86b814db9bb6 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:13.824016 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa19271-648e-4114-a360-62490fc583a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:13.825195 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa19271-648e-4114-a360-62490fc583a1 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:13.825579 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa19271-648e-4114-a360-62490fc583a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:13.826095 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa19271-648e-4114-a360-62490fc583a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:13.826915 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa19271-648e-4114-a360-62490fc583a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:13.827289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa19271-648e-4114-a360-62490fc583a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:13.829154 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa19271-648e-4114-a360-62490fc583a1 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:13.845481 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 357/709] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:13.847128 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa19271-648e-4114-a360-62490fc583a1 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:13.866610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c02cb-3818-4db7-bfed-bb9e76f9f5cc None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:13.903773 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:13.912273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53152cce-72b6-42ac-a05d-ae0d9539d2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:13.913477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53152cce-72b6-42ac-a05d-ae0d9539d2c9 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:13.913772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53152cce-72b6-42ac-a05d-ae0d9539d2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:13.914050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53152cce-72b6-42ac-a05d-ae0d9539d2c9 None admin] PATH_INFO: `/v3/services/network` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:13.914646 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53152cce-72b6-42ac-a05d-ae0d9539d2c9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:13.916081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53152cce-72b6-42ac-a05d-ae0d9539d2c9 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:13.932077 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-53152cce-72b6-42ac-a05d-ae0d9539d2c9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 04:15:13.932077 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:13.932077 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 04:15:13.932077 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 04:15:13.932077 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 04:15:13.932077 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 04:15:13.932077 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 04:15:13.932077 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:13.932077 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 04:15:13.932077 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:13.932077 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:15:13.932077 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:13.932077 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:13.932077 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:13.932077 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:13.932077 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:13.932077 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:13.933096 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:13.933096 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:13.933096 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:13.933096 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:13.933096 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:13.933096 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:13.933096 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 04:15:13.933096 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 04:15:13.933096 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 04:15:13.933096 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 04:15:13.933096 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:13.933096 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:13.933096 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:13.933096 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:13.933096 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:13.933096 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:13.933921 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:13.933921 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:13.933921 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:13.933921 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:13.933921 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:13.933921 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:13.933921 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:13.933921 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:13.933921 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:13.933921 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:13.933921 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 04:15:13.933921 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:13.933921 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 04:15:13.933921 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:13.933921 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 353/710] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:15:13 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:13.934691 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53152cce-72b6-42ac-a05d-ae0d9539d2c9 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:13.969591 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa19271-648e-4114-a360-62490fc583a1 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:13.972931 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:14.007052 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505eb89-e785-4100-bdf3-fef4060c4f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:14.007686 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505eb89-e785-4100-bdf3-fef4060c4f18 None admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:14.007998 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505eb89-e785-4100-bdf3-fef4060c4f18 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:14.008436 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505eb89-e785-4100-bdf3-fef4060c4f18 None admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:14.009560 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3505eb89-e785-4100-bdf3-fef4060c4f18 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:14.011861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3505eb89-e785-4100-bdf3-fef4060c4f18 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:14.039993 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 358/711] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Aug 23 04:15:13 2022] POST /identity/v3/users => generated 254 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:15:14.039993 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505eb89-e785-4100-bdf3-fef4060c4f18 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:14.060361 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53152cce-72b6-42ac-a05d-ae0d9539d2c9 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:14.102534 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0397bf-dfcc-43ca-9a0f-a7b901908429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:14.103323 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0397bf-dfcc-43ca-9a0f-a7b901908429 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:14.103672 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0397bf-dfcc-43ca-9a0f-a7b901908429 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:14.104062 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0397bf-dfcc-43ca-9a0f-a7b901908429 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:14.113992 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0397bf-dfcc-43ca-9a0f-a7b901908429 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:14.115827 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0397bf-dfcc-43ca-9a0f-a7b901908429 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:14.128966 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-bb0397bf-dfcc-43ca-9a0f-a7b901908429 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:14.128966 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:14.128966 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:14.128966 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:14.128966 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:14.128966 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:14.128966 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:14.128966 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:14.128966 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:14.128966 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:14.128966 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:14.128966 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:14.128966 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:15:14.128966 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:15:14.128966 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:15:14.130112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:15:14.130112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:14.130112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:14.130112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:14.130112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:14.130112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:14.130112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:14.130112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:14.130112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:14.130112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:14.130112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:14.130112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:14.130112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:14.130112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:14.130112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:14.130112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:14.130112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:14.131165 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:15:14.131165 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:15:14.131165 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:15:14.131165 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:15:14.131165 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:15:14.131165 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:15:14.131165 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:14.131165 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:14.133851 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 354/712] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:15:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:14.135012 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0397bf-dfcc-43ca-9a0f-a7b901908429 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:14.196372 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:14.202627 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505eb89-e785-4100-bdf3-fef4060c4f18 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:14.251252 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0d52a-ba3b-44e3-a8ce-5abd261da039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:14.252210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0d52a-ba3b-44e3-a8ce-5abd261da039 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:14.253463 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0d52a-ba3b-44e3-a8ce-5abd261da039 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:14.253915 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0d52a-ba3b-44e3-a8ce-5abd261da039 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:14.254852 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c0d52a-ba3b-44e3-a8ce-5abd261da039 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:14.257033 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c0d52a-ba3b-44e3-a8ce-5abd261da039 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:14.271955 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:14.273684 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-d1c0d52a-ba3b-44e3-a8ce-5abd261da039 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 04:15:14.273684 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:14.273684 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 04:15:14.273684 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 04:15:14.273684 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 04:15:14.273684 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 04:15:14.273684 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 04:15:14.273684 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:14.273684 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 04:15:14.273684 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:14.273684 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:15:14.273684 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:14.273684 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:14.273684 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:14.273684 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:14.273684 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:14.273684 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:14.275349 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:14.275349 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:14.275349 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:14.275349 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:14.275349 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:14.275349 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:14.275349 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 04:15:14.275349 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 04:15:14.275349 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 04:15:14.275349 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 04:15:14.275349 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:14.275349 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:14.275349 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:14.275349 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:14.275349 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:14.275349 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:14.276374 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:14.276374 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:14.276374 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:14.276374 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:14.276374 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:14.276374 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:14.276374 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:14.276374 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:14.276374 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:14.276374 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:14.276374 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 04:15:14.276374 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:14.276374 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 04:15:14.276374 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:14.276374 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 359/713] 10.222.0.18 () {56 vars in 1129 bytes} [Tue Aug 23 04:15:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:14.277544 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0d52a-ba3b-44e3-a8ce-5abd261da039 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:14.278508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0397bf-dfcc-43ca-9a0f-a7b901908429 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:14.323756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4961bcd-459d-4e9d-86a6-1eb32b104c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:14.324478 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4961bcd-459d-4e9d-86a6-1eb32b104c5f None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:14.324761 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4961bcd-459d-4e9d-86a6-1eb32b104c5f None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:14.325045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4961bcd-459d-4e9d-86a6-1eb32b104c5f None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:14.325735 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4961bcd-459d-4e9d-86a6-1eb32b104c5f None admin] RBAC: Adding query filter params (name=network) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:14.326049 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4961bcd-459d-4e9d-86a6-1eb32b104c5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:14.327453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4961bcd-459d-4e9d-86a6-1eb32b104c5f None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:14.335863 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 355/714] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Aug 23 04:15:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:14.336592 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4961bcd-459d-4e9d-86a6-1eb32b104c5f None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:14.426132 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0d52a-ba3b-44e3-a8ce-5abd261da039 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:14.469611 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d6ea4-3a21-45a3-b1f4-2c720b6227c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:14.470242 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d6ea4-3a21-45a3-b1f4-2c720b6227c4 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:14.470523 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d6ea4-3a21-45a3-b1f4-2c720b6227c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:14.470801 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d6ea4-3a21-45a3-b1f4-2c720b6227c4 None admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:14.471510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d6ea4-3a21-45a3-b1f4-2c720b6227c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:14.471818 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d6ea4-3a21-45a3-b1f4-2c720b6227c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:14.473307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d6ea4-3a21-45a3-b1f4-2c720b6227c4 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:14.487042 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 360/715] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:15:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:14.487947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d6ea4-3a21-45a3-b1f4-2c720b6227c4 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:14.506355 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4961bcd-459d-4e9d-86a6-1eb32b104c5f None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:14.535580 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:14.558232 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1bccc3-512a-40f9-ae83-427488e4d147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:14.563060 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bccc3-512a-40f9-ae83-427488e4d147 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:14.563060 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bccc3-512a-40f9-ae83-427488e4d147 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:14.563060 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bccc3-512a-40f9-ae83-427488e4d147 None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:14.563060 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1bccc3-512a-40f9-ae83-427488e4d147 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:14.563060 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1bccc3-512a-40f9-ae83-427488e4d147 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:14.563060 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1bccc3-512a-40f9-ae83-427488e4d147 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:14.573051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 356/716] 10.222.0.18 () {56 vars in 1124 bytes} [Tue Aug 23 04:15:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:14.573051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1bccc3-512a-40f9-ae83-427488e4d147 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:14.601527 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:14.655770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d6ea4-3a21-45a3-b1f4-2c720b6227c4 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:14.697565 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1bccc3-512a-40f9-ae83-427488e4d147 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:14.703734 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcef49c-f497-45bf-8a07-7b61ab043cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:14.704627 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcef49c-f497-45bf-8a07-7b61ab043cba None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:14.705076 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcef49c-f497-45bf-8a07-7b61ab043cba None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:14.705562 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcef49c-f497-45bf-8a07-7b61ab043cba None admin] PATH_INFO: `/v3/services` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:14.706377 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcef49c-f497-45bf-8a07-7b61ab043cba None admin] RBAC: Adding query filter params (type=network) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:14.707053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcef49c-f497-45bf-8a07-7b61ab043cba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:14.709679 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcef49c-f497-45bf-8a07-7b61ab043cba None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:14.717374 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 361/717] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Aug 23 04:15:14 2022] GET /identity/v3/services?type=network => generated 124 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:14.718672 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcef49c-f497-45bf-8a07-7b61ab043cba None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:14.743669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eaf7cb-b719-4a78-b2d7-4d5b580b27b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:14.743669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eaf7cb-b719-4a78-b2d7-4d5b580b27b6 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:14.743669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eaf7cb-b719-4a78-b2d7-4d5b580b27b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:14.743669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eaf7cb-b719-4a78-b2d7-4d5b580b27b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:14.747346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eaf7cb-b719-4a78-b2d7-4d5b580b27b6 None admin] RBAC: Adding query filter params (role.id=52b31061b72c45ccb76dd26bf31f8d3f, scope.project.id=9f4373dfb5f9425eb73669e24d18a08f, user.id=cd7b11f479a54adc8d9b8641859b07a1) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:14.747346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eaf7cb-b719-4a78-b2d7-4d5b580b27b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:14.747346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eaf7cb-b719-4a78-b2d7-4d5b580b27b6 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:14.755369 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 357/718] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:15:14 2022] GET /identity/v3/role_assignments?user.id=cd7b11f479a54adc8d9b8641859b07a1&scope.project.id=9f4373dfb5f9425eb73669e24d18a08f&role.id=52b31061b72c45ccb76dd26bf31f8d3f => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:14.907122 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:14.945737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcef49c-f497-45bf-8a07-7b61ab043cba None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:15.006202 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9a35e-8297-4a4e-9c97-78a8acc0181a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:15.007262 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9a35e-8297-4a4e-9c97-78a8acc0181a None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:15.007744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9a35e-8297-4a4e-9c97-78a8acc0181a None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:15.008307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9a35e-8297-4a4e-9c97-78a8acc0181a None admin] PATH_INFO: `/v3/services` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:15.009456 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a9a35e-8297-4a4e-9c97-78a8acc0181a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:15.010434 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a9a35e-8297-4a4e-9c97-78a8acc0181a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:15.012239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a9a35e-8297-4a4e-9c97-78a8acc0181a None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:15.016096 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87e152-fb69-41d3-9982-35b55c99189c None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:15.016482 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87e152-fb69-41d3-9982-35b55c99189c None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:15.016631 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87e152-fb69-41d3-9982-35b55c99189c None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:15.018213 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 358/719] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:15.019493 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 362/720] 10.222.0.18 () {56 vars in 1124 bytes} [Tue Aug 23 04:15:14 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:15.024631 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fd729-e52e-4270-9d41-0be48581bec5 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:15.025233 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fd729-e52e-4270-9d41-0be48581bec5 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:15.025736 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fd729-e52e-4270-9d41-0be48581bec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:15.065132 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-2f7fd729-e52e-4270-9d41-0be48581bec5 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:15.066759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-2f7fd729-e52e-4270-9d41-0be48581bec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:15.226818 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 359/721] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:15:15.578022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9a35e-8297-4a4e-9c97-78a8acc0181a None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:15.731864 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9a35e-8297-4a4e-9c97-78a8acc0181a None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:15.777791 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd78d6-2845-477d-8935-61bc31d51062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:15.778440 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd78d6-2845-477d-8935-61bc31d51062 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:15.778722 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd78d6-2845-477d-8935-61bc31d51062 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:15.778998 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd78d6-2845-477d-8935-61bc31d51062 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:15.798741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd78d6-2845-477d-8935-61bc31d51062 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:15:15.804024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-04bd78d6-2845-477d-8935-61bc31d51062 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:15:15.804024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:15.804024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:15.804024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:15.804024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:15.804024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:15.804024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:15.804024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:15.804024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:15.804024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:15.804024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:15.804024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:15.804024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:15:15.804024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:15:15.804024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:15:15.804024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:15:15.804909 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:15:15.804909 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:15:15.804909 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:15:15.804909 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:15:15.804909 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:15.804909 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:15.804909 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:15.804909 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:15.804909 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:15.804909 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:15.804909 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:15.804909 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:15.804909 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:15.804909 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:15.804909 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:15.804909 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:15.806289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:15.806289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:15.806289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:15.806289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:15.806289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:15:15.806289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:15:15.806289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:15:15.806289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:15:15.806289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:15:15.806289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:15:15.806289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:15:15.806289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:15.806289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 363/722] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:15:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:15.811033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fd729-e52e-4270-9d41-0be48581bec5 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:15.839696 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:15.949919 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fd729-e52e-4270-9d41-0be48581bec5 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:16.014085 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c361f5-af86-4c51-806f-48bf5215a8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:16.015645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c361f5-af86-4c51-806f-48bf5215a8cd None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:16.016188 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c361f5-af86-4c51-806f-48bf5215a8cd None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:16.016669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c361f5-af86-4c51-806f-48bf5215a8cd None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:16.017585 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c361f5-af86-4c51-806f-48bf5215a8cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:16.018095 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c361f5-af86-4c51-806f-48bf5215a8cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:16.019664 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c361f5-af86-4c51-806f-48bf5215a8cd None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:16.034572 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 360/723] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:15:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:16.043738 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd78d6-2845-477d-8935-61bc31d51062 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:16.067794 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:16.222854 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd78d6-2845-477d-8935-61bc31d51062 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:16.270327 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413ce59-9443-43da-872f-b05d908b0286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:16.271496 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413ce59-9443-43da-872f-b05d908b0286 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:16.272607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413ce59-9443-43da-872f-b05d908b0286 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:16.273099 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413ce59-9443-43da-872f-b05d908b0286 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:16.291732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8413ce59-9443-43da-872f-b05d908b0286 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:16.293640 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8413ce59-9443-43da-872f-b05d908b0286 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:16.312903 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-8413ce59-9443-43da-872f-b05d908b0286 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:16.312903 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:16.312903 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:16.312903 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:16.312903 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:16.312903 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:16.312903 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:16.312903 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:16.312903 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:16.312903 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:16.312903 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:16.312903 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:16.312903 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:16.312903 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:16.312903 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:16.312903 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:16.313917 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:16.313917 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:16.313917 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:16.313917 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:16.313917 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:16.313917 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:16.313917 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:16.313917 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:16.313917 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:16.313917 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:16.313917 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:16.313917 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:16.313917 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:16.313917 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:16.313917 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:16.313917 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:16.313917 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:16.314767 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:16.314767 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:16.314767 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:16.316761 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 364/724] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:16.322952 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c361f5-af86-4c51-806f-48bf5215a8cd None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:16.423983 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:16.471496 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c361f5-af86-4c51-806f-48bf5215a8cd None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:16.516673 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b26afc-ceb1-4cb1-b140-d7b25bdadb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:16.517211 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b26afc-ceb1-4cb1-b140-d7b25bdadb51 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:16.517493 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b26afc-ceb1-4cb1-b140-d7b25bdadb51 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:16.517768 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b26afc-ceb1-4cb1-b140-d7b25bdadb51 None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:16.518419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b26afc-ceb1-4cb1-b140-d7b25bdadb51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:16.518722 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b26afc-ceb1-4cb1-b140-d7b25bdadb51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:16.520222 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b26afc-ceb1-4cb1-b140-d7b25bdadb51 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:16.538792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 361/725] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:16.544429 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413ce59-9443-43da-872f-b05d908b0286 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:16.640624 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:16.687132 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413ce59-9443-43da-872f-b05d908b0286 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:16.731851 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a12fec-5895-450c-8ae3-751f961e6224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:16.732514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a12fec-5895-450c-8ae3-751f961e6224 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:16.732786 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a12fec-5895-450c-8ae3-751f961e6224 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:16.733052 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a12fec-5895-450c-8ae3-751f961e6224 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:16.742516 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a12fec-5895-450c-8ae3-751f961e6224 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:16.743990 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a12fec-5895-450c-8ae3-751f961e6224 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:16.757048 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-43a12fec-5895-450c-8ae3-751f961e6224 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 04:15:16.757048 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:16.757048 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:16.757048 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:16.757048 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:16.757048 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:16.757048 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:16.757048 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:16.757048 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:16.757048 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:16.757048 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:16.757048 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:16.757048 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:15:16.757048 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:15:16.757048 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:15:16.757048 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:15:16.758005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:16.758005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:16.758005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:15:16.758005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:16.758005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:15:16.758005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:16.758005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:16.758005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:16.758005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:16.758005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:16.758005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:16.758005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:16.758005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:16.758005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:16.758005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:16.758005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:16.758005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:16.758005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:16.758926 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:16.758926 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:16.758926 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:15:16.758926 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:15:16.758926 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:15:16.758926 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:15:16.758926 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:15:16.758926 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:15:16.758926 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 04:15:16.758926 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:16.758926 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 365/726] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:15:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:16.764118 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b26afc-ceb1-4cb1-b140-d7b25bdadb51 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:16.900086 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b26afc-ceb1-4cb1-b140-d7b25bdadb51 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:16.933489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e26b91-ae6e-4567-ac88-182957d5e16b None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:16.933837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e26b91-ae6e-4567-ac88-182957d5e16b None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:16.934036 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e26b91-ae6e-4567-ac88-182957d5e16b None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:16.935593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 366/727] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:16.949658 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac5792-ca44-4233-b42d-5b1da6578598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:16.950283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac5792-ca44-4233-b42d-5b1da6578598 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:16.950558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac5792-ca44-4233-b42d-5b1da6578598 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:16.950827 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac5792-ca44-4233-b42d-5b1da6578598 None admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:16.951564 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac5792-ca44-4233-b42d-5b1da6578598 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:16.951864 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac5792-ca44-4233-b42d-5b1da6578598 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:16.953340 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac5792-ca44-4233-b42d-5b1da6578598 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:16.959711 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952b8e5-d605-4f5a-8bc5-cf9ffacce179 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:16.960086 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952b8e5-d605-4f5a-8bc5-cf9ffacce179 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:16.960371 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952b8e5-d605-4f5a-8bc5-cf9ffacce179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:16.970761 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:16.982474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-a952b8e5-d605-4f5a-8bc5-cf9ffacce179 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:16.983684 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a952b8e5-d605-4f5a-8bc5-cf9ffacce179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:16.989440 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 362/728] 10.222.0.18 () {56 vars in 1135 bytes} [Tue Aug 23 04:15:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:16.995430 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac5792-ca44-4233-b42d-5b1da6578598 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:17.066030 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 367/729] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:15:17.168847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac5792-ca44-4233-b42d-5b1da6578598 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:17.220903 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d8f16-641f-4419-a317-e8fa3c415521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:17.221521 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d8f16-641f-4419-a317-e8fa3c415521 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:17.221798 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d8f16-641f-4419-a317-e8fa3c415521 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:17.222071 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d8f16-641f-4419-a317-e8fa3c415521 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:17.237129 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60981f1-efdf-47bf-b60e-8f2e72d7b2cc None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:17.237441 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60981f1-efdf-47bf-b60e-8f2e72d7b2cc None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:17.237693 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60981f1-efdf-47bf-b60e-8f2e72d7b2cc None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:17.239328 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 368/730] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:17.250038 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569be48-efc7-4430-b346-e2d144f2d556 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:17.250475 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569be48-efc7-4430-b346-e2d144f2d556 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:17.250908 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569be48-efc7-4430-b346-e2d144f2d556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:17.288319 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:17.293506 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-f569be48-efc7-4430-b346-e2d144f2d556 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:17.294059 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0d8f16-641f-4419-a317-e8fa3c415521 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:17.296704 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-f569be48-efc7-4430-b346-e2d144f2d556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:17.296810 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0d8f16-641f-4419-a317-e8fa3c415521 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:17.398328 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 363/731] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:15:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 04:15:17.403971 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d8f16-641f-4419-a317-e8fa3c415521 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:17.417068 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:17.441092 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 369/732] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:15:17.442775 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff2dc3-453d-4224-9b72-103ab6357ef6 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:17.443445 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff2dc3-453d-4224-9b72-103ab6357ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:17.444226 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff2dc3-453d-4224-9b72-103ab6357ef6 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:17.466136 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 370/733] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:17.467403 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff2dc3-453d-4224-9b72-103ab6357ef6 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:17.508383 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:17.514101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d8f16-641f-4419-a317-e8fa3c415521 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:17.549657 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ed5b4-758a-4593-9e38-c85507a02099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:17.550289 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ed5b4-758a-4593-9e38-c85507a02099 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:17.550563 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ed5b4-758a-4593-9e38-c85507a02099 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:17.550830 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ed5b4-758a-4593-9e38-c85507a02099 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:17.560965 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843ed5b4-758a-4593-9e38-c85507a02099 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:17.562439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843ed5b4-758a-4593-9e38-c85507a02099 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:17.578417 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-843ed5b4-758a-4593-9e38-c85507a02099 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:17.578417 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:17.578417 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:17.578417 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:17.578417 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:17.578417 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:17.578417 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:17.578417 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:17.578417 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:17.578417 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:17.578417 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:17.578417 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:17.578417 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:17.578417 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:17.578417 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:17.578417 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:17.579365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:17.579365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:17.579365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:17.579365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:17.579365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:17.579365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:17.579365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:17.579365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:17.579365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:17.579365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:17.579365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:17.579365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:17.579365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:17.579365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:17.579365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:17.579365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:17.579365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:17.580281 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:17.580281 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:17.580281 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:17.580281 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 364/734] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:15:17.581088 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4be25-a781-4990-a0ad-13e09ea0949c None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:17.581088 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4be25-a781-4990-a0ad-13e09ea0949c None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:17.581327 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4be25-a781-4990-a0ad-13e09ea0949c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:17.587674 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff2dc3-453d-4224-9b72-103ab6357ef6 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:17.599848 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-83c4be25-a781-4990-a0ad-13e09ea0949c None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:17.601040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-83c4be25-a781-4990-a0ad-13e09ea0949c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:17.626509 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498614ad-38b1-48ba-9759-857f089d2526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:17.627359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498614ad-38b1-48ba-9759-857f089d2526 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:17.627643 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498614ad-38b1-48ba-9759-857f089d2526 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:17.627937 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498614ad-38b1-48ba-9759-857f089d2526 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:17.638911 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498614ad-38b1-48ba-9759-857f089d2526 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:15:17.643510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-498614ad-38b1-48ba-9759-857f089d2526 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:15:17.643510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:17.643510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:17.643510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:17.643510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:17.643510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:17.643510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:17.643510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:17.643510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:17.643510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:17.643510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:17.643510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:17.643510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:15:17.643510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:15:17.643510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:15:17.643510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:15:17.644871 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:15:17.644871 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:15:17.644871 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:15:17.644871 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:15:17.644871 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:17.644871 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:17.644871 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:17.644871 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:17.644871 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:17.644871 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:17.644871 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:17.644871 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:17.644871 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:17.644871 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:17.644871 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:17.644871 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:17.645761 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:17.645761 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:17.645761 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:17.645761 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:17.645761 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:15:17.645761 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:15:17.645761 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:15:17.645761 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:15:17.645761 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:15:17.645761 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:15:17.645761 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:15:17.645761 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:17.645761 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 371/735] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Aug 23 04:15:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:17.646574 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb5e25-3cbf-4e2d-b995-402fadb3f791 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:17.647580 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb5e25-3cbf-4e2d-b995-402fadb3f791 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:17.647580 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb5e25-3cbf-4e2d-b995-402fadb3f791 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:17.648878 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 372/736] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:17.649585 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb5e25-3cbf-4e2d-b995-402fadb3f791 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:17.679527 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 365/737] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:15:17.680267 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4be25-a781-4990-a0ad-13e09ea0949c None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:17.709090 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:17.763298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:17.777413 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb5e25-3cbf-4e2d-b995-402fadb3f791 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:17.787377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4be25-a781-4990-a0ad-13e09ea0949c None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:17.814439 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c454e2-350e-41e7-9d6b-4429e5d8b665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:17.815182 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c454e2-350e-41e7-9d6b-4429e5d8b665 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:17.815457 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c454e2-350e-41e7-9d6b-4429e5d8b665 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:17.815735 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c454e2-350e-41e7-9d6b-4429e5d8b665 None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:17.816522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c454e2-350e-41e7-9d6b-4429e5d8b665 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:17.816995 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c454e2-350e-41e7-9d6b-4429e5d8b665 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:17.818447 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c454e2-350e-41e7-9d6b-4429e5d8b665 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:17.825997 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e280f4-cc14-4724-aeba-9e4f50476878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:17.826622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e280f4-cc14-4724-aeba-9e4f50476878 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:17.826902 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e280f4-cc14-4724-aeba-9e4f50476878 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:17.827180 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e280f4-cc14-4724-aeba-9e4f50476878 None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:17.827855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e280f4-cc14-4724-aeba-9e4f50476878 None admin] RBAC: Adding query filter params (name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:17.828193 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e280f4-cc14-4724-aeba-9e4f50476878 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:17.829565 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e280f4-cc14-4724-aeba-9e4f50476878 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:17.832614 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 373/738] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:17.834017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca78067-6c33-40a0-865b-4da2abac3752 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:17.834456 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca78067-6c33-40a0-865b-4da2abac3752 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:17.834958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca78067-6c33-40a0-865b-4da2abac3752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:17.841143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 366/739] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Aug 23 04:15:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:17.842131 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e280f4-cc14-4724-aeba-9e4f50476878 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:17.859672 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-fca78067-6c33-40a0-865b-4da2abac3752 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:17.860908 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-fca78067-6c33-40a0-865b-4da2abac3752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:17.944078 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 374/740] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 04:15:17.944956 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca78067-6c33-40a0-865b-4da2abac3752 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:17.946942 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e280f4-cc14-4724-aeba-9e4f50476878 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:17.979945 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:17.987263 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92830fe5-1aff-49ca-89ee-430e1f808fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:17.987908 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92830fe5-1aff-49ca-89ee-430e1f808fda None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:17.988215 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92830fe5-1aff-49ca-89ee-430e1f808fda None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:17.988490 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92830fe5-1aff-49ca-89ee-430e1f808fda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:17.996355 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:18.000118 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92830fe5-1aff-49ca-89ee-430e1f808fda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:15:18.005309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-92830fe5-1aff-49ca-89ee-430e1f808fda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:15:18.005309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:18.005309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:18.005309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:18.005309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:18.005309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:18.005309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:18.005309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:18.005309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:18.005309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:18.005309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:18.005309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:18.005309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:15:18.005309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:15:18.005309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:15:18.005309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:15:18.006723 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:15:18.006723 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:15:18.006723 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:15:18.006723 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:15:18.006723 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:18.006723 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:18.006723 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:18.006723 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:18.006723 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:18.006723 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:18.006723 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:18.006723 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:18.006723 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:18.006723 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:18.006723 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:18.006723 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:18.007715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:18.007715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:18.007715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:18.007715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:18.007715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:15:18.007715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:15:18.007715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:15:18.007715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:15:18.007715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:15:18.007715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:15:18.007715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:15:18.007715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:18.011886 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 367/741] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:15:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:18.012545 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92830fe5-1aff-49ca-89ee-430e1f808fda None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:18.060392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca78067-6c33-40a0-865b-4da2abac3752 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:18.097953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd382ee-69c4-47be-b9d1-b2310f020f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:18.098928 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd382ee-69c4-47be-b9d1-b2310f020f11 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:18.099422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd382ee-69c4-47be-b9d1-b2310f020f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:18.099811 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd382ee-69c4-47be-b9d1-b2310f020f11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:18.106719 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd382ee-69c4-47be-b9d1-b2310f020f11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:18.108308 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd382ee-69c4-47be-b9d1-b2310f020f11 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:18.121747 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92830fe5-1aff-49ca-89ee-430e1f808fda None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:18.123255 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-ecd382ee-69c4-47be-b9d1-b2310f020f11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:18.123255 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:18.123255 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:18.123255 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:18.123255 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:18.123255 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:18.123255 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:18.123255 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:18.123255 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:18.123255 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:18.123255 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:18.123255 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:18.123255 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:15:18.123255 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:15:18.123255 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:15:18.124107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:15:18.124107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:18.124107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:18.124107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:18.124107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:18.124107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:18.124107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:18.124107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:18.124107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:18.124107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:18.124107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:18.124107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:18.124107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:18.124107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:18.124107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:18.124107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:18.124107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:18.124966 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:15:18.124966 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:15:18.124966 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:15:18.124966 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:15:18.124966 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:15:18.124966 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:15:18.124966 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:18.124966 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:18.130686 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 375/742] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:15:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:18.130686 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd382ee-69c4-47be-b9d1-b2310f020f11 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:18.157942 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a877308-6d3b-4b16-a2ae-7cc024be8b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:18.158832 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a877308-6d3b-4b16-a2ae-7cc024be8b05 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:18.159291 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a877308-6d3b-4b16-a2ae-7cc024be8b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:18.159560 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a877308-6d3b-4b16-a2ae-7cc024be8b05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:18.171548 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a877308-6d3b-4b16-a2ae-7cc024be8b05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:18.173033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a877308-6d3b-4b16-a2ae-7cc024be8b05 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:18.187522 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-1a877308-6d3b-4b16-a2ae-7cc024be8b05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:18.187522 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:18.187522 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:18.187522 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:18.187522 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:18.187522 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:18.187522 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:18.187522 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:18.187522 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:18.187522 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:18.187522 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:18.187522 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:18.187522 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:18.187522 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:18.187522 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:18.187522 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:18.188520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:18.188520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:18.188520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:18.188520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:18.188520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:18.188520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:18.188520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:18.188520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:18.188520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:18.188520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:18.188520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:18.188520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:18.188520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:18.188520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:18.188520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:18.188520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:18.188520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:18.189404 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:18.189404 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:18.189404 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:18.193509 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 368/743] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:18.194352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a877308-6d3b-4b16-a2ae-7cc024be8b05 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:18.225561 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:18.234961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd382ee-69c4-47be-b9d1-b2310f020f11 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:18.247174 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:18.268921 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172ae4e-d91d-47a3-8cc4-353490b4a899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:18.269533 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172ae4e-d91d-47a3-8cc4-353490b4a899 None admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:18.269807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172ae4e-d91d-47a3-8cc4-353490b4a899 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:18.270073 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172ae4e-d91d-47a3-8cc4-353490b4a899 None admin] PATH_INFO: `/v3/services` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:18.270710 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172ae4e-d91d-47a3-8cc4-353490b4a899 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:18.272294 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172ae4e-d91d-47a3-8cc4-353490b4a899 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:18.289859 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 376/744] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Aug 23 04:15:18 2022] POST /identity/v3/services => generated 240 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:15:18.290904 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172ae4e-d91d-47a3-8cc4-353490b4a899 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:18.298785 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a877308-6d3b-4b16-a2ae-7cc024be8b05 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:18.352697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963a40c-510d-4927-8962-692f2a52add5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:18.353328 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963a40c-510d-4927-8962-692f2a52add5 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:18.353614 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963a40c-510d-4927-8962-692f2a52add5 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:18.353886 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963a40c-510d-4927-8962-692f2a52add5 None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:18.354573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8963a40c-510d-4927-8962-692f2a52add5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:18.354878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8963a40c-510d-4927-8962-692f2a52add5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:18.356497 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8963a40c-510d-4927-8962-692f2a52add5 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:18.372044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 369/745] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:15:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:18.372044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963a40c-510d-4927-8962-692f2a52add5 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:18.426777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172ae4e-d91d-47a3-8cc4-353490b4a899 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:18.472333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45394149-fab7-4f11-bb97-efb9794c7e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:18.472948 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45394149-fab7-4f11-bb97-efb9794c7e83 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:18.473223 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45394149-fab7-4f11-bb97-efb9794c7e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:18.473488 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45394149-fab7-4f11-bb97-efb9794c7e83 None admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:18.474189 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45394149-fab7-4f11-bb97-efb9794c7e83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:18.474759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45394149-fab7-4f11-bb97-efb9794c7e83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:18.476269 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45394149-fab7-4f11-bb97-efb9794c7e83 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:18.487175 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963a40c-510d-4927-8962-692f2a52add5 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:18.496045 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 377/746] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:15:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:18.496045 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45394149-fab7-4f11-bb97-efb9794c7e83 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:18.499336 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:18.513032 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:18.525935 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce191ac-0718-4c5d-9ba9-6910ce36ed2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:18.526552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce191ac-0718-4c5d-9ba9-6910ce36ed2b None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:18.526888 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce191ac-0718-4c5d-9ba9-6910ce36ed2b None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:18.527165 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce191ac-0718-4c5d-9ba9-6910ce36ed2b None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:18.527813 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce191ac-0718-4c5d-9ba9-6910ce36ed2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:18.528141 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce191ac-0718-4c5d-9ba9-6910ce36ed2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:18.529564 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce191ac-0718-4c5d-9ba9-6910ce36ed2b None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:18.542209 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 370/747] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:18.543049 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce191ac-0718-4c5d-9ba9-6910ce36ed2b None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:18.607125 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45394149-fab7-4f11-bb97-efb9794c7e83 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:18.645263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c760bdf-c641-42ca-8c2a-dc35da21be60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:18.645954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c760bdf-c641-42ca-8c2a-dc35da21be60 None admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:18.646278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c760bdf-c641-42ca-8c2a-dc35da21be60 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:18.646445 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c760bdf-c641-42ca-8c2a-dc35da21be60 None admin] PATH_INFO: `/v3/services` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:18.647111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c760bdf-c641-42ca-8c2a-dc35da21be60 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:18.648519 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c760bdf-c641-42ca-8c2a-dc35da21be60 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:18.657259 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce191ac-0718-4c5d-9ba9-6910ce36ed2b None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:18.664547 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 378/748] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Aug 23 04:15:18 2022] POST /identity/v3/services => generated 251 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:15:18.665564 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c760bdf-c641-42ca-8c2a-dc35da21be60 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:18.716882 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf918c-da94-4e91-a1b1-45014c06dd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:18.717842 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf918c-da94-4e91-a1b1-45014c06dd8a None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:18.718271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf918c-da94-4e91-a1b1-45014c06dd8a None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:18.718744 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf918c-da94-4e91-a1b1-45014c06dd8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:18.742054 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf918c-da94-4e91-a1b1-45014c06dd8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:18.743667 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf918c-da94-4e91-a1b1-45014c06dd8a None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:18.765214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-abbf918c-da94-4e91-a1b1-45014c06dd8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:18.765214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:18.765214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:18.765214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:18.765214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:18.765214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:18.765214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:18.765214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:18.765214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:18.765214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:18.765214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:18.765214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:18.765214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:18.765214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:18.765214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:18.765214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:18.766399 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:18.766399 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:18.766399 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:18.766399 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:18.766399 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:18.766399 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:18.766399 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:18.766399 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:18.766399 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:18.766399 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:18.766399 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:18.766399 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:18.766399 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:18.766399 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:18.766399 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:18.766399 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:18.766399 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:18.767482 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:18.767482 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:18.767482 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:18.769203 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 371/749] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:18.770782 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:18.772563 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf918c-da94-4e91-a1b1-45014c06dd8a None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:18.800054 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:18.823872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c760bdf-c641-42ca-8c2a-dc35da21be60 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:18.870357 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97768137-a002-40d4-9394-5fdb1b969924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:18.871143 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97768137-a002-40d4-9394-5fdb1b969924 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:18.871509 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97768137-a002-40d4-9394-5fdb1b969924 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:18.871870 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97768137-a002-40d4-9394-5fdb1b969924 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:18.872811 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97768137-a002-40d4-9394-5fdb1b969924 None admin] RBAC: Adding query filter params (role.id=52b31061b72c45ccb76dd26bf31f8d3f, scope.project.id=9f4373dfb5f9425eb73669e24d18a08f, user.id=3f1418fd211a448da1a50175dce13b32) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:18.873209 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97768137-a002-40d4-9394-5fdb1b969924 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:18.874830 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97768137-a002-40d4-9394-5fdb1b969924 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:18.882929 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 379/750] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:15:18 2022] GET /identity/v3/role_assignments?user.id=3f1418fd211a448da1a50175dce13b32&scope.project.id=9f4373dfb5f9425eb73669e24d18a08f&role.id=52b31061b72c45ccb76dd26bf31f8d3f => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:18.887999 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97768137-a002-40d4-9394-5fdb1b969924 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:18.909122 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf918c-da94-4e91-a1b1-45014c06dd8a None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:18.960776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f55cbe-d295-4d4b-b06a-4d87be881fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:18.961458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f55cbe-d295-4d4b-b06a-4d87be881fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:18.961718 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f55cbe-d295-4d4b-b06a-4d87be881fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:18.961996 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f55cbe-d295-4d4b-b06a-4d87be881fc5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:18.978207 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f55cbe-d295-4d4b-b06a-4d87be881fc5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:18.979792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f55cbe-d295-4d4b-b06a-4d87be881fc5 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:19.005475 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-d6f55cbe-d295-4d4b-b06a-4d87be881fc5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 04:15:19.005475 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:19.005475 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:19.005475 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:19.005475 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:19.005475 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:19.005475 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:19.005475 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:19.005475 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:19.005475 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:19.005475 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:19.005475 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:19.005475 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:15:19.005475 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:15:19.005475 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:15:19.005475 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:15:19.006679 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:19.006679 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:19.006679 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:15:19.006679 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:19.006679 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:15:19.006679 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:19.006679 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:19.006679 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:19.006679 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:19.006679 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:19.006679 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:19.006679 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:19.006679 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:19.006679 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:19.006679 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:19.006679 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:19.006679 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:19.006679 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:19.007762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:19.007762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:19.007762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:15:19.007762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:15:19.007762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:15:19.007762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:15:19.007762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:15:19.007762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:15:19.007762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 04:15:19.007762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:19.011011 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 372/751] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Aug 23 04:15:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:19.013970 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f55cbe-d295-4d4b-b06a-4d87be881fc5 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:19.030816 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97768137-a002-40d4-9394-5fdb1b969924 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:19.070573 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925852c4-6c3f-4e66-8964-a2b2c1440ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:19.071501 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925852c4-6c3f-4e66-8964-a2b2c1440ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:19.072526 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925852c4-6c3f-4e66-8964-a2b2c1440ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:19.072991 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925852c4-6c3f-4e66-8964-a2b2c1440ce3 None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:19.074109 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925852c4-6c3f-4e66-8964-a2b2c1440ce3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:19.074608 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925852c4-6c3f-4e66-8964-a2b2c1440ce3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:19.077098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925852c4-6c3f-4e66-8964-a2b2c1440ce3 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:19.081255 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:19.102935 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 380/752] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:19.104840 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925852c4-6c3f-4e66-8964-a2b2c1440ce3 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:19.146010 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:19.188845 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f55cbe-d295-4d4b-b06a-4d87be881fc5 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:19.246476 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272b9fde-8c5d-49d5-9a69-f478656b35a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:19.247101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b9fde-8c5d-49d5-9a69-f478656b35a8 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:19.247379 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b9fde-8c5d-49d5-9a69-f478656b35a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:19.247706 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b9fde-8c5d-49d5-9a69-f478656b35a8 None admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:19.248475 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272b9fde-8c5d-49d5-9a69-f478656b35a8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:19.248782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272b9fde-8c5d-49d5-9a69-f478656b35a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:19.250227 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272b9fde-8c5d-49d5-9a69-f478656b35a8 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:19.264879 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925852c4-6c3f-4e66-8964-a2b2c1440ce3 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:19.271136 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 373/753] 10.222.0.18 () {56 vars in 1149 bytes} [Tue Aug 23 04:15:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:19.276553 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272b9fde-8c5d-49d5-9a69-f478656b35a8 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:19.345230 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cedea74-5d6d-41d8-b5a0-95b8f0fd010c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:19.345880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cedea74-5d6d-41d8-b5a0-95b8f0fd010c None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:19.346153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cedea74-5d6d-41d8-b5a0-95b8f0fd010c None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:19.346417 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cedea74-5d6d-41d8-b5a0-95b8f0fd010c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:19.358160 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cedea74-5d6d-41d8-b5a0-95b8f0fd010c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:19.359832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cedea74-5d6d-41d8-b5a0-95b8f0fd010c None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:19.382532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-6cedea74-5d6d-41d8-b5a0-95b8f0fd010c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 04:15:19.382532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:19.382532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:19.382532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:19.382532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:19.382532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:19.382532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:19.382532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:19.382532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:19.382532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:19.382532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:19.382532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:19.382532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:15:19.382532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:15:19.382532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:15:19.382532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:15:19.383520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:19.383520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:19.383520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:15:19.383520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:19.383520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:15:19.383520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:19.383520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:19.383520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:19.383520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:19.383520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:19.383520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:19.383520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:19.383520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:19.383520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:19.383520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:19.383520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:19.383520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:19.383520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:19.384471 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:19.384471 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:19.384471 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:15:19.384471 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:15:19.384471 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:15:19.384471 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:15:19.384471 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:15:19.384471 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:15:19.384471 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 04:15:19.384471 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:19.389551 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 381/754] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Aug 23 04:15:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:19.392076 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cedea74-5d6d-41d8-b5a0-95b8f0fd010c None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:19.460119 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272b9fde-8c5d-49d5-9a69-f478656b35a8 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:19.475342 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:19.500119 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e009f7-783f-4e17-9c19-27e9fb0fe505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:19.501095 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e009f7-783f-4e17-9c19-27e9fb0fe505 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:19.501620 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e009f7-783f-4e17-9c19-27e9fb0fe505 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:19.502104 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e009f7-783f-4e17-9c19-27e9fb0fe505 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:19.532503 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:19.569660 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cedea74-5d6d-41d8-b5a0-95b8f0fd010c None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:19.592521 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e009f7-783f-4e17-9c19-27e9fb0fe505 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:19.594308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e009f7-783f-4e17-9c19-27e9fb0fe505 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:19.670716 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966d723-73a3-4039-9f73-bb2232858666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:19.672080 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966d723-73a3-4039-9f73-bb2232858666 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:19.672560 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966d723-73a3-4039-9f73-bb2232858666 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:19.674492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966d723-73a3-4039-9f73-bb2232858666 None admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:19.675456 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f966d723-73a3-4039-9f73-bb2232858666 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:19.680890 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f966d723-73a3-4039-9f73-bb2232858666 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:19.682558 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f966d723-73a3-4039-9f73-bb2232858666 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:19.720214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 382/755] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Aug 23 04:15:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:19.721168 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966d723-73a3-4039-9f73-bb2232858666 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:19.726430 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 374/756] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:15:19 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 04:15:19.732106 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e009f7-783f-4e17-9c19-27e9fb0fe505 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:19.776295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:19.891388 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966d723-73a3-4039-9f73-bb2232858666 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:19.902176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e009f7-783f-4e17-9c19-27e9fb0fe505 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:19.908400 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:19.938868 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966dd9c2-793e-40c3-a55a-8b4daff6fe4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:19.939511 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966dd9c2-793e-40c3-a55a-8b4daff6fe4e None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:19.939788 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966dd9c2-793e-40c3-a55a-8b4daff6fe4e None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:19.940093 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966dd9c2-793e-40c3-a55a-8b4daff6fe4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:19.961094 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966dd9c2-793e-40c3-a55a-8b4daff6fe4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:19.962547 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966dd9c2-793e-40c3-a55a-8b4daff6fe4e None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:19.982669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-966dd9c2-793e-40c3-a55a-8b4daff6fe4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:19.982669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:19.982669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:19.982669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:19.982669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:19.982669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:19.982669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:19.982669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:19.982669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:19.982669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:19.982669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:19.982669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:19.982669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:19.982669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:19.982669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:19.982669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:19.983607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:19.983607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:19.983607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:19.983607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:19.983607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:19.983607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:19.983607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:19.983607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:19.983607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:19.983607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:19.983607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:19.983607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:19.983607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:19.983607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:19.983607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:19.983607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:19.983607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:19.984509 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:19.984509 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:19.984509 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:19.988035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 383/757] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:19.989813 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966dd9c2-793e-40c3-a55a-8b4daff6fe4e None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:19.997573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a83014-c3f0-47ff-9c2b-043b0d0f0034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:19.998238 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a83014-c3f0-47ff-9c2b-043b0d0f0034 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:19.998522 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a83014-c3f0-47ff-9c2b-043b0d0f0034 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:19.998800 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a83014-c3f0-47ff-9c2b-043b0d0f0034 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:20.018731 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a83014-c3f0-47ff-9c2b-043b0d0f0034 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:20.020232 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a83014-c3f0-47ff-9c2b-043b0d0f0034 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:20.034759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-f5a83014-c3f0-47ff-9c2b-043b0d0f0034 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:20.034759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:20.034759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:20.034759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:20.034759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:20.034759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:20.034759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:20.034759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:20.034759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:20.034759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:20.034759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:20.034759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:20.034759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:20.034759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:20.034759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:20.034759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:20.036636 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:20.036636 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:20.036636 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:20.036636 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:20.036636 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:20.036636 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:20.036636 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:20.036636 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:20.036636 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:20.036636 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:20.036636 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:20.036636 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:20.036636 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:20.036636 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:20.036636 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:20.036636 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:20.036636 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:20.037904 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:20.037904 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:20.037904 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:20.037904 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 375/758] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:15:20.043519 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a83014-c3f0-47ff-9c2b-043b0d0f0034 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:20.158802 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966dd9c2-793e-40c3-a55a-8b4daff6fe4e None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:20.181227 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a83014-c3f0-47ff-9c2b-043b0d0f0034 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:20.184263 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:20.219505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369c8d83-9684-4679-9a86-446241b563ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:20.220384 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c8d83-9684-4679-9a86-446241b563ed None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:20.220740 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c8d83-9684-4679-9a86-446241b563ed None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:20.221095 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c8d83-9684-4679-9a86-446241b563ed None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:20.221882 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369c8d83-9684-4679-9a86-446241b563ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:20.222251 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369c8d83-9684-4679-9a86-446241b563ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:20.224437 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369c8d83-9684-4679-9a86-446241b563ed None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:20.237588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 376/759] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:20.242892 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369c8d83-9684-4679-9a86-446241b563ed None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:20.300659 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf5139-7dfe-487b-a4d2-eca322e9253e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:20.301295 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf5139-7dfe-487b-a4d2-eca322e9253e None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:20.301573 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf5139-7dfe-487b-a4d2-eca322e9253e None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:20.301841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf5139-7dfe-487b-a4d2-eca322e9253e None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:20.302546 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf5139-7dfe-487b-a4d2-eca322e9253e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:20.302862 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf5139-7dfe-487b-a4d2-eca322e9253e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:20.304277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf5139-7dfe-487b-a4d2-eca322e9253e None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:20.342188 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 384/760] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:20.354027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369c8d83-9684-4679-9a86-446241b563ed None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:20.362612 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf5139-7dfe-487b-a4d2-eca322e9253e None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:20.393635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce128f-4a37-4dae-bdfb-f6967379da69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:20.394705 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce128f-4a37-4dae-bdfb-f6967379da69 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:20.394989 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce128f-4a37-4dae-bdfb-f6967379da69 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:20.395265 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce128f-4a37-4dae-bdfb-f6967379da69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:20.402996 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce128f-4a37-4dae-bdfb-f6967379da69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:20.404622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce128f-4a37-4dae-bdfb-f6967379da69 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:20.417026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-6bce128f-4a37-4dae-bdfb-f6967379da69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:20.417026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:20.417026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:20.417026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:20.417026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:20.417026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:20.417026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:20.417026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:20.417026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:20.417026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:20.417026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:20.417026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:20.417026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:15:20.417026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:15:20.417026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:15:20.418772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:15:20.418772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:20.418772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:20.418772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:20.418772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:20.418772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:20.418772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:20.418772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:20.418772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:20.418772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:20.418772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:20.418772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:20.418772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:20.418772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:20.418772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:20.418772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:20.418772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:20.419832 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:15:20.419832 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:15:20.419832 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:15:20.419832 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:15:20.419832 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:15:20.419832 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:15:20.419832 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:20.419832 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:20.419832 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 377/761] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:15:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:20.426157 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce128f-4a37-4dae-bdfb-f6967379da69 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:20.449974 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:20.464407 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:20.514732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf5139-7dfe-487b-a4d2-eca322e9253e None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:20.557011 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a194eb-4c20-4b90-b7a6-140f1b82355f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:20.557644 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a194eb-4c20-4b90-b7a6-140f1b82355f None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:20.557918 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a194eb-4c20-4b90-b7a6-140f1b82355f None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:20.558184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a194eb-4c20-4b90-b7a6-140f1b82355f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:20.565273 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a194eb-4c20-4b90-b7a6-140f1b82355f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:20.566702 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a194eb-4c20-4b90-b7a6-140f1b82355f None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:20.578240 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-a5a194eb-4c20-4b90-b7a6-140f1b82355f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:20.578240 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:20.578240 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:20.578240 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:20.578240 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:20.578240 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:20.578240 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:20.578240 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:20.578240 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:20.578240 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:20.578240 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:20.578240 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:20.578240 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:15:20.578240 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:15:20.578240 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:15:20.579254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:15:20.579254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:20.579254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:20.579254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:20.579254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:20.579254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:20.579254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:20.579254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:20.579254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:20.579254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:20.579254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:20.579254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:20.579254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:20.579254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:20.579254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:20.579254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:20.579254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:20.580289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:15:20.580289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:15:20.580289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:15:20.580289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:15:20.580289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:15:20.580289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:15:20.580289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:20.580289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:20.580289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 385/762] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:15:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:20.580950 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb1e23-ed57-456f-9b11-d9d536f452c3 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:20.581235 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb1e23-ed57-456f-9b11-d9d536f452c3 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:20.582307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb1e23-ed57-456f-9b11-d9d536f452c3 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:20.583948 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 386/763] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:20.587885 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb1e23-ed57-456f-9b11-d9d536f452c3 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:20.604479 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce128f-4a37-4dae-bdfb-f6967379da69 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:20.642656 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0ae8b-ddae-4cf7-b921-e4a560ad8884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:20.643278 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0ae8b-ddae-4cf7-b921-e4a560ad8884 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:20.643566 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0ae8b-ddae-4cf7-b921-e4a560ad8884 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:20.643845 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0ae8b-ddae-4cf7-b921-e4a560ad8884 None admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:20.644747 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0ae8b-ddae-4cf7-b921-e4a560ad8884 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:20.644936 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0ae8b-ddae-4cf7-b921-e4a560ad8884 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:20.646507 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0ae8b-ddae-4cf7-b921-e4a560ad8884 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:20.659678 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 378/764] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:15:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:20.661027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1590ab-ba71-4c69-8363-e9edc78aaee5 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:20.661330 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1590ab-ba71-4c69-8363-e9edc78aaee5 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:20.661610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1590ab-ba71-4c69-8363-e9edc78aaee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:20.680314 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-2b1590ab-ba71-4c69-8363-e9edc78aaee5 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:20.681431 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-2b1590ab-ba71-4c69-8363-e9edc78aaee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:20.683264 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:20.701947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb1e23-ed57-456f-9b11-d9d536f452c3 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:20.738415 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8610c-6e6a-4e1e-a7b8-940bf1357086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:20.739280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8610c-6e6a-4e1e-a7b8-940bf1357086 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:20.739726 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8610c-6e6a-4e1e-a7b8-940bf1357086 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:20.740256 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8610c-6e6a-4e1e-a7b8-940bf1357086 None admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:20.741107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8610c-6e6a-4e1e-a7b8-940bf1357086 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:20.741555 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8610c-6e6a-4e1e-a7b8-940bf1357086 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:20.743201 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8610c-6e6a-4e1e-a7b8-940bf1357086 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:20.748853 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:20.756465 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 387/766] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:15:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:20.757625 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8610c-6e6a-4e1e-a7b8-940bf1357086 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:20.759999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 379/766] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:15:20.763369 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1590ab-ba71-4c69-8363-e9edc78aaee5 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:20.911122 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8610c-6e6a-4e1e-a7b8-940bf1357086 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:20.913291 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1590ab-ba71-4c69-8363-e9edc78aaee5 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:20.947394 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f1cd54-b825-4a5b-86c0-73ebf6facedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:20.948281 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1cd54-b825-4a5b-86c0-73ebf6facedc None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:20.948695 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1cd54-b825-4a5b-86c0-73ebf6facedc None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:20.949098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1cd54-b825-4a5b-86c0-73ebf6facedc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:20.949967 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f1cd54-b825-4a5b-86c0-73ebf6facedc None admin] RBAC: Adding query filter params (role.id=bdf71ef6ab10430e8c356cd4d952ff5c, scope.project.id=9f4373dfb5f9425eb73669e24d18a08f, user.id=d3538271ebfd41aabe9f3da85feb7bca) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:20.950426 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f1cd54-b825-4a5b-86c0-73ebf6facedc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:20.952014 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f1cd54-b825-4a5b-86c0-73ebf6facedc None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:20.956872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723e9b8-ce3f-44e4-984b-d1dbb2123aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:20.957647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723e9b8-ce3f-44e4-984b-d1dbb2123aca None admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:20.958019 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723e9b8-ce3f-44e4-984b-d1dbb2123aca None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:20.958388 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723e9b8-ce3f-44e4-984b-d1dbb2123aca None admin] PATH_INFO: `/v3/projects/9f4373dfb5f9425eb73669e24d18a08f/users/cd7b11f479a54adc8d9b8641859b07a1/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:20.959062 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 388/767] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:15:20 2022] GET /identity/v3/role_assignments?user.id=d3538271ebfd41aabe9f3da85feb7bca&scope.project.id=9f4373dfb5f9425eb73669e24d18a08f&role.id=bdf71ef6ab10430e8c356cd4d952ff5c => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:20.960423 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a40ba-97db-48ab-ad02-403a0674932b None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:20.960845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a40ba-97db-48ab-ad02-403a0674932b None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:20.960956 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6723e9b8-ce3f-44e4-984b-d1dbb2123aca None admin] RBAC: Authorizing `identity:create_grant(project_id=9f4373dfb5f9425eb73669e24d18a08f, user_id=cd7b11f479a54adc8d9b8641859b07a1, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:20.961367 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a40ba-97db-48ab-ad02-403a0674932b None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:20.962620 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6723e9b8-ce3f-44e4-984b-d1dbb2123aca None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:20.962954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 389/768] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:20.974300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb439be-fe46-4d12-a2c4-b71be9c7d2d9 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:20.974870 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb439be-fe46-4d12-a2c4-b71be9c7d2d9 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:20.975274 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb439be-fe46-4d12-a2c4-b71be9c7d2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:20.998186 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 380/769] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Aug 23 04:15:20 2022] PUT /identity/v3/projects/9f4373dfb5f9425eb73669e24d18a08f/users/cd7b11f479a54adc8d9b8641859b07a1/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:15:21.003811 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-abb439be-fe46-4d12-a2c4-b71be9c7d2d9 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:21.005214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-abb439be-fe46-4d12-a2c4-b71be9c7d2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:21.008941 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:21.092101 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 390/770] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:15:21.100170 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723e9b8-ce3f-44e4-984b-d1dbb2123aca None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:21.174053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d144ed-ffd3-497e-a994-d41745d0eae5 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:21.174722 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d144ed-ffd3-497e-a994-d41745d0eae5 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:21.175198 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d144ed-ffd3-497e-a994-d41745d0eae5 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:21.177133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 391/771] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:15:21.180319 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:21.183598 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4242c6d-5b85-41ad-b928-2dc0cfb664ad None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:21.184269 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4242c6d-5b85-41ad-b928-2dc0cfb664ad None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:21.184786 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4242c6d-5b85-41ad-b928-2dc0cfb664ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:21.211832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-e4242c6d-5b85-41ad-b928-2dc0cfb664ad None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:21.213679 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-e4242c6d-5b85-41ad-b928-2dc0cfb664ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:21.295874 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 392/772] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 04:15:21.378953 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723e9b8-ce3f-44e4-984b-d1dbb2123aca None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:21.429745 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d3397-8d8e-47a7-9a3b-3af85e580de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:21.430701 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d3397-8d8e-47a7-9a3b-3af85e580de9 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:21.431160 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d3397-8d8e-47a7-9a3b-3af85e580de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:21.431608 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d3397-8d8e-47a7-9a3b-3af85e580de9 None admin] PATH_INFO: `/v3/services/network` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:21.432363 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240d3397-8d8e-47a7-9a3b-3af85e580de9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:21.433882 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240d3397-8d8e-47a7-9a3b-3af85e580de9 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:21.440198 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4242c6d-5b85-41ad-b928-2dc0cfb664ad None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:21.449787 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-240d3397-8d8e-47a7-9a3b-3af85e580de9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 04:15:21.449787 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:21.449787 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 04:15:21.449787 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 04:15:21.449787 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 04:15:21.449787 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 04:15:21.449787 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 04:15:21.449787 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:21.449787 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 04:15:21.449787 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:21.449787 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:15:21.449787 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:21.449787 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:21.449787 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:21.449787 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:21.449787 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:21.449787 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:21.450987 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:21.450987 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:21.450987 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:21.450987 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:21.450987 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:21.450987 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:21.450987 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 04:15:21.450987 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 04:15:21.450987 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 04:15:21.450987 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 04:15:21.450987 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:21.450987 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:21.450987 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:21.450987 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:21.450987 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:21.450987 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:21.451889 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:21.451889 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:21.451889 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:21.451889 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:21.451889 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:21.451889 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:21.451889 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:21.451889 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:21.451889 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:21.451889 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:21.451889 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 04:15:21.451889 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:21.451889 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 04:15:21.451889 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:21.454221 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 381/773] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:15:21 2022] GET /identity/v3/services/network => generated 88 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:21.468590 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d3397-8d8e-47a7-9a3b-3af85e580de9 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:21.584800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:21.626205 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4242c6d-5b85-41ad-b928-2dc0cfb664ad None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:21.632179 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d3397-8d8e-47a7-9a3b-3af85e580de9 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:21.662676 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:21.669935 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed000e-d31d-4025-bbf3-0297f62ca452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:21.671546 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed000e-d31d-4025-bbf3-0297f62ca452 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:21.672683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed000e-d31d-4025-bbf3-0297f62ca452 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:21.673316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed000e-d31d-4025-bbf3-0297f62ca452 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:21.675679 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed000e-d31d-4025-bbf3-0297f62ca452 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:21.677534 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed000e-d31d-4025-bbf3-0297f62ca452 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:21.680804 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a1c84-7151-4727-a3f3-96017d0d267d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:21.681662 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a1c84-7151-4727-a3f3-96017d0d267d None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:21.682147 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a1c84-7151-4727-a3f3-96017d0d267d None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:21.682612 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a1c84-7151-4727-a3f3-96017d0d267d None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:21.683460 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a1c84-7151-4727-a3f3-96017d0d267d None admin] RBAC: Adding query filter params (name=network) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:21.683988 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a1c84-7151-4727-a3f3-96017d0d267d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:21.685642 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a1c84-7151-4727-a3f3-96017d0d267d None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:21.687031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-0fed000e-d31d-4025-bbf3-0297f62ca452 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 04:15:21.687031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:21.687031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 04:15:21.687031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 04:15:21.687031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 04:15:21.687031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 04:15:21.687031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 04:15:21.687031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:21.687031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 04:15:21.687031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:21.687031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:15:21.687031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:21.687031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:21.687031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:21.687031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:21.687031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:21.687031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:21.688343 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:21.688343 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:21.688343 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:21.688343 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:21.688343 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:21.688343 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:21.688343 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 04:15:21.688343 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 04:15:21.688343 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 04:15:21.688343 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 04:15:21.688343 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:21.688343 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:21.688343 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:21.688343 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:21.688343 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:21.688343 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:21.689150 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:21.689150 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:21.689150 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:21.689150 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:21.689150 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:21.689150 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:21.689150 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:21.689150 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:21.689150 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:21.689150 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:21.689150 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 04:15:21.689150 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:21.689150 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 04:15:21.689150 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:21.689150 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 393/774] 10.222.0.18 () {56 vars in 1129 bytes} [Tue Aug 23 04:15:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:15:21.689952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed000e-d31d-4025-bbf3-0297f62ca452 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:21.700453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 382/775] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Aug 23 04:15:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:21.701298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a1c84-7151-4727-a3f3-96017d0d267d None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:21.801732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed000e-d31d-4025-bbf3-0297f62ca452 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:21.813068 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a1c84-7151-4727-a3f3-96017d0d267d None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:21.837117 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f99c71-8235-4a04-ae6c-9cfbabb50ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:21.838503 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f99c71-8235-4a04-ae6c-9cfbabb50ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:21.839025 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f99c71-8235-4a04-ae6c-9cfbabb50ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:21.839507 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f99c71-8235-4a04-ae6c-9cfbabb50ce4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:21.841710 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:21.852447 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5467f3-8c7e-41fe-a459-7489f949dac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:21.852947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f99c71-8235-4a04-ae6c-9cfbabb50ce4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:15:21.853261 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5467f3-8c7e-41fe-a459-7489f949dac0 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:21.853804 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5467f3-8c7e-41fe-a459-7489f949dac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:21.854084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5467f3-8c7e-41fe-a459-7489f949dac0 None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:21.854721 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5467f3-8c7e-41fe-a459-7489f949dac0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:21.855020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5467f3-8c7e-41fe-a459-7489f949dac0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:21.856614 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5467f3-8c7e-41fe-a459-7489f949dac0 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:21.859427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-a0f99c71-8235-4a04-ae6c-9cfbabb50ce4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:15:21.859427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:21.859427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:21.859427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:21.859427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:21.859427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:21.859427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:21.859427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:21.859427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:21.859427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:21.859427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:21.859427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:21.859427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:15:21.859427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:15:21.859427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:15:21.859427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:15:21.860522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:15:21.860522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:15:21.860522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:15:21.860522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:15:21.860522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:21.860522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:21.860522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:21.860522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:21.860522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:21.860522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:21.860522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:21.860522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:21.860522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:21.860522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:21.860522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:21.860522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:21.861453 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:21.861453 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:21.861453 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:21.861453 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:21.861453 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:15:21.861453 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:15:21.861453 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:15:21.861453 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:15:21.861453 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:15:21.861453 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:15:21.861453 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:15:21.861453 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:21.863984 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 394/776] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:15:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:21.865102 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f99c71-8235-4a04-ae6c-9cfbabb50ce4 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:21.870748 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 383/777] 10.222.0.18 () {56 vars in 1124 bytes} [Tue Aug 23 04:15:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:21.871673 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5467f3-8c7e-41fe-a459-7489f949dac0 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:21.926122 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:21.969421 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f99c71-8235-4a04-ae6c-9cfbabb50ce4 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:21.981875 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5467f3-8c7e-41fe-a459-7489f949dac0 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:22.004987 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b65fe0-f33d-447e-84a1-a39d44c48a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:22.005970 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b65fe0-f33d-447e-84a1-a39d44c48a59 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:22.006524 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b65fe0-f33d-447e-84a1-a39d44c48a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:22.007014 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b65fe0-f33d-447e-84a1-a39d44c48a59 None admin] PATH_INFO: `/v3/services` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:22.007842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b65fe0-f33d-447e-84a1-a39d44c48a59 None admin] RBAC: Adding query filter params (type=network) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:22.008404 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b65fe0-f33d-447e-84a1-a39d44c48a59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:22.010119 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b65fe0-f33d-447e-84a1-a39d44c48a59 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:22.015772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 395/778] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Aug 23 04:15:21 2022] GET /identity/v3/services?type=network => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:22.016800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b65fe0-f33d-447e-84a1-a39d44c48a59 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:22.023530 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c318e67-4b99-4d93-be5d-67821feba8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:22.024184 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c318e67-4b99-4d93-be5d-67821feba8dd None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:22.024461 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c318e67-4b99-4d93-be5d-67821feba8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:22.024730 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c318e67-4b99-4d93-be5d-67821feba8dd None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:22.025407 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c318e67-4b99-4d93-be5d-67821feba8dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:22.025698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c318e67-4b99-4d93-be5d-67821feba8dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:22.027062 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c318e67-4b99-4d93-be5d-67821feba8dd None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:22.041248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 384/779] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:15:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:22.042138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c318e67-4b99-4d93-be5d-67821feba8dd None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:22.080668 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:22.125541 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b65fe0-f33d-447e-84a1-a39d44c48a59 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:22.153189 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c318e67-4b99-4d93-be5d-67821feba8dd None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:22.161114 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800ea55-e4b8-4b0d-97e3-311f0795e134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:22.162138 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800ea55-e4b8-4b0d-97e3-311f0795e134 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:22.162626 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800ea55-e4b8-4b0d-97e3-311f0795e134 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:22.163104 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800ea55-e4b8-4b0d-97e3-311f0795e134 None admin] PATH_INFO: `/v3/services` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:22.163967 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9800ea55-e4b8-4b0d-97e3-311f0795e134 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:22.164470 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9800ea55-e4b8-4b0d-97e3-311f0795e134 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:22.166112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9800ea55-e4b8-4b0d-97e3-311f0795e134 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:22.167042 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:22.172211 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 396/780] 10.222.0.18 () {56 vars in 1124 bytes} [Tue Aug 23 04:15:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:22.173382 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800ea55-e4b8-4b0d-97e3-311f0795e134 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:22.193339 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc83b5-a3e3-41f3-b558-5887e15e8aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:22.193957 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc83b5-a3e3-41f3-b558-5887e15e8aa2 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:22.194356 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc83b5-a3e3-41f3-b558-5887e15e8aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:22.194576 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc83b5-a3e3-41f3-b558-5887e15e8aa2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:22.195299 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc83b5-a3e3-41f3-b558-5887e15e8aa2 None admin] RBAC: Adding query filter params (interface=public, service_id=9f165f88de614a0f93fbf93510d3fd30, region_id=RegionOne) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:22.195597 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc83b5-a3e3-41f3-b558-5887e15e8aa2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:22.197112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc83b5-a3e3-41f3-b558-5887e15e8aa2 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:22.210298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 385/781] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Aug 23 04:15:22 2022] GET /identity/v3/endpoints?service_id=9f165f88de614a0f93fbf93510d3fd30&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:22.211313 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc83b5-a3e3-41f3-b558-5887e15e8aa2 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:22.282951 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800ea55-e4b8-4b0d-97e3-311f0795e134 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:22.306594 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:22.319761 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7864b90-0712-435d-b701-0d4ae1393777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:22.321040 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7864b90-0712-435d-b701-0d4ae1393777 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:22.321395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc83b5-a3e3-41f3-b558-5887e15e8aa2 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:22.321782 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7864b90-0712-435d-b701-0d4ae1393777 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:22.322251 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7864b90-0712-435d-b701-0d4ae1393777 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:22.335411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7864b90-0712-435d-b701-0d4ae1393777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:22.337414 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7864b90-0712-435d-b701-0d4ae1393777 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:22.352874 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-b7864b90-0712-435d-b701-0d4ae1393777 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:22.352874 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:22.352874 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:22.352874 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:22.352874 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:22.352874 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:22.352874 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:22.352874 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:22.352874 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:22.352874 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:22.352874 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:22.352874 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:22.352874 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:22.352874 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:22.352874 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:22.352874 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:22.353776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:22.353776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:22.353776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:22.353776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:22.353776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:22.353776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:22.353776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:22.353776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:22.353776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:22.353776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:22.353776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:22.353776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:22.353776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:22.353776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:22.353776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:22.353776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:22.353776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:22.354657 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:22.354657 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:22.354657 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:22.356708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 397/782] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:22.357721 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7864b90-0712-435d-b701-0d4ae1393777 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:22.366703 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30c11e-a2aa-4412-a27b-74ada64353c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:22.367443 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30c11e-a2aa-4412-a27b-74ada64353c5 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:22.367723 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30c11e-a2aa-4412-a27b-74ada64353c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:22.368029 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30c11e-a2aa-4412-a27b-74ada64353c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:22.368797 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae30c11e-a2aa-4412-a27b-74ada64353c5 None admin] RBAC: Adding query filter params (interface=public, service_id=1cbb8cc81ca84579bfb32984c6490467, region_id=RegionOne) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:22.369108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae30c11e-a2aa-4412-a27b-74ada64353c5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:22.370600 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae30c11e-a2aa-4412-a27b-74ada64353c5 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:22.376792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 386/783] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Aug 23 04:15:22 2022] GET /identity/v3/endpoints?service_id=1cbb8cc81ca84579bfb32984c6490467&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:22.377775 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30c11e-a2aa-4412-a27b-74ada64353c5 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:22.428944 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:22.479091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7864b90-0712-435d-b701-0d4ae1393777 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:22.498975 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30c11e-a2aa-4412-a27b-74ada64353c5 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:22.515277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03dcfd-682a-4e07-823e-b52302377a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:22.516294 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03dcfd-682a-4e07-823e-b52302377a98 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:22.516765 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03dcfd-682a-4e07-823e-b52302377a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:22.517220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03dcfd-682a-4e07-823e-b52302377a98 None admin] PATH_INFO: `/v3/services` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:22.518097 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03dcfd-682a-4e07-823e-b52302377a98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:22.519674 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03dcfd-682a-4e07-823e-b52302377a98 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:22.525576 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 398/784] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Aug 23 04:15:22 2022] GET /identity/v3/services => generated 772 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:22.526616 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03dcfd-682a-4e07-823e-b52302377a98 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:22.554532 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2f1e7-230f-44df-bcbc-b8d41fbe0de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:22.555362 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2f1e7-230f-44df-bcbc-b8d41fbe0de3 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:22.555541 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2f1e7-230f-44df-bcbc-b8d41fbe0de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:22.555809 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2f1e7-230f-44df-bcbc-b8d41fbe0de3 None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:22.556501 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e2f1e7-230f-44df-bcbc-b8d41fbe0de3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:22.556843 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e2f1e7-230f-44df-bcbc-b8d41fbe0de3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:22.558336 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e2f1e7-230f-44df-bcbc-b8d41fbe0de3 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:22.572376 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:22.584687 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 387/785] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:22.591180 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2f1e7-230f-44df-bcbc-b8d41fbe0de3 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:22.639670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03dcfd-682a-4e07-823e-b52302377a98 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:22.679897 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62489461-0a22-4448-813c-6f1a9a61959e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:22.680930 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62489461-0a22-4448-813c-6f1a9a61959e None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:22.681402 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62489461-0a22-4448-813c-6f1a9a61959e None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:22.681942 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62489461-0a22-4448-813c-6f1a9a61959e None admin] PATH_INFO: `/v3/services` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:22.682872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62489461-0a22-4448-813c-6f1a9a61959e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:22.684508 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62489461-0a22-4448-813c-6f1a9a61959e None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:22.690864 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 399/786] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Aug 23 04:15:22 2022] GET /identity/v3/services => generated 772 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:22.763568 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:22.772506 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2f1e7-230f-44df-bcbc-b8d41fbe0de3 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:22.827460 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d792419-927e-4624-b8e7-bd4b4e38a380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:22.828207 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d792419-927e-4624-b8e7-bd4b4e38a380 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:22.828499 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d792419-927e-4624-b8e7-bd4b4e38a380 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:22.828822 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d792419-927e-4624-b8e7-bd4b4e38a380 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:22.838906 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d792419-927e-4624-b8e7-bd4b4e38a380 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:22.840429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d792419-927e-4624-b8e7-bd4b4e38a380 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:22.856032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-0d792419-927e-4624-b8e7-bd4b4e38a380 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 04:15:22.856032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:22.856032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:22.856032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:22.856032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:22.856032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:22.856032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:22.856032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:22.856032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:22.856032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:22.856032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:22.856032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:22.856032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:15:22.856032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:15:22.856032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:15:22.856032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:15:22.856986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:22.856986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:22.856986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:15:22.856986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:22.856986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:15:22.856986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:22.856986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:22.856986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:22.856986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:22.856986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:22.856986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:22.856986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:22.856986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:22.856986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:22.856986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:22.856986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:22.856986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:22.856986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:22.857906 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:22.857906 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:22.857906 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:15:22.857906 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:15:22.857906 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:15:22.857906 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:15:22.857906 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:15:22.857906 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:15:22.857906 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 04:15:22.857906 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:22.857906 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 388/787] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:15:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:22.863553 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62489461-0a22-4448-813c-6f1a9a61959e None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:22.986719 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62489461-0a22-4448-813c-6f1a9a61959e None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:22.988714 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:23.023356 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70591ad2-ce9e-4edc-b485-267f72f9f3dc None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:23.024050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70591ad2-ce9e-4edc-b485-267f72f9f3dc None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:23.024982 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70591ad2-ce9e-4edc-b485-267f72f9f3dc None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:23.026811 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 389/788] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:23.031447 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b376dbe-2aff-4579-9829-44f337d927d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:23.032111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b376dbe-2aff-4579-9829-44f337d927d9 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:23.032394 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b376dbe-2aff-4579-9829-44f337d927d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:23.032662 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b376dbe-2aff-4579-9829-44f337d927d9 None admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:23.033069 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e5727-6490-4c3d-911d-a56f7af95eaa None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:23.033403 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b376dbe-2aff-4579-9829-44f337d927d9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:23.033704 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b376dbe-2aff-4579-9829-44f337d927d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:23.033960 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e5727-6490-4c3d-911d-a56f7af95eaa None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:23.034436 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e5727-6490-4c3d-911d-a56f7af95eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:23.035132 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b376dbe-2aff-4579-9829-44f337d927d9 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:23.063616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-838e5727-6490-4c3d-911d-a56f7af95eaa None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:23.064867 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-838e5727-6490-4c3d-911d-a56f7af95eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:23.069727 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 400/789] 10.222.0.18 () {56 vars in 1135 bytes} [Tue Aug 23 04:15:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:23.076142 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b376dbe-2aff-4579-9829-44f337d927d9 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:23.164398 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 390/790] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 04:15:23.252128 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8712541-67e1-43fa-ad89-ca4b2caf6b08 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:23.252458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8712541-67e1-43fa-ad89-ca4b2caf6b08 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:23.252680 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8712541-67e1-43fa-ad89-ca4b2caf6b08 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:23.254297 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 391/791] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:15:23.256534 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b376dbe-2aff-4579-9829-44f337d927d9 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:23.259542 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca98d7-9104-42fb-b354-5791e0c88b4a None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:23.259827 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca98d7-9104-42fb-b354-5791e0c88b4a None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:23.260131 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca98d7-9104-42fb-b354-5791e0c88b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:23.279380 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-00ca98d7-9104-42fb-b354-5791e0c88b4a None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:23.281466 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-00ca98d7-9104-42fb-b354-5791e0c88b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:23.320549 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6475eea-b960-4842-9fcb-b2ebf3de3ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:23.321191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6475eea-b960-4842-9fcb-b2ebf3de3ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:23.321465 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6475eea-b960-4842-9fcb-b2ebf3de3ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:23.321732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6475eea-b960-4842-9fcb-b2ebf3de3ab1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:23.341627 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6475eea-b960-4842-9fcb-b2ebf3de3ab1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:23.343269 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6475eea-b960-4842-9fcb-b2ebf3de3ab1 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:23.373798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-c6475eea-b960-4842-9fcb-b2ebf3de3ab1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:23.373798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:23.373798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:23.373798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:23.373798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:23.373798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:23.373798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:23.373798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:23.373798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:23.373798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:23.373798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:23.373798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:23.373798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:23.373798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:23.373798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:23.373798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:23.374794 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:23.374794 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:23.374794 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:23.374794 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:23.374794 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:23.374794 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:23.374794 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:23.374794 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:23.374794 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:23.374794 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:23.374794 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:23.374794 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:23.374794 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:23.374794 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:23.374794 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:23.374794 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:23.374794 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:23.375709 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:23.375709 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:23.375709 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:23.375709 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 401/792] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:15:23.381529 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6475eea-b960-4842-9fcb-b2ebf3de3ab1 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:23.403623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 392/793] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 04:15:23.420558 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:23.481218 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca98d7-9104-42fb-b354-5791e0c88b4a None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:23.508691 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:23.547178 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6475eea-b960-4842-9fcb-b2ebf3de3ab1 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:23.601595 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f7b7d2-b75b-4d98-bcac-304f597550a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:23.602576 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7b7d2-b75b-4d98-bcac-304f597550a3 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:23.603072 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7b7d2-b75b-4d98-bcac-304f597550a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:23.603568 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7b7d2-b75b-4d98-bcac-304f597550a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:23.604450 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f7b7d2-b75b-4d98-bcac-304f597550a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:23.604969 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f7b7d2-b75b-4d98-bcac-304f597550a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:23.606545 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f7b7d2-b75b-4d98-bcac-304f597550a3 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:23.621998 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 402/794] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:23.632496 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f7b7d2-b75b-4d98-bcac-304f597550a3 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:23.638709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca98d7-9104-42fb-b354-5791e0c88b4a None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:23.684347 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051aba25-3230-43f8-8203-31ee3ba46b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:23.684901 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051aba25-3230-43f8-8203-31ee3ba46b68 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:23.685175 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051aba25-3230-43f8-8203-31ee3ba46b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:23.685449 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051aba25-3230-43f8-8203-31ee3ba46b68 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:23.702920 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051aba25-3230-43f8-8203-31ee3ba46b68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:15:23.707497 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-051aba25-3230-43f8-8203-31ee3ba46b68 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:15:23.707497 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:23.707497 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:23.707497 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:23.707497 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:23.707497 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:23.707497 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:23.707497 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:23.707497 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:23.707497 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:23.707497 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:23.707497 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:23.707497 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:15:23.707497 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:15:23.707497 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:15:23.707497 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:15:23.708469 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:15:23.708469 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:15:23.708469 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:15:23.708469 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:15:23.708469 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:23.708469 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:23.708469 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:23.708469 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:23.708469 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:23.708469 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:23.708469 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:23.708469 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:23.708469 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:23.708469 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:23.708469 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:23.708469 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:23.709910 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:23.709910 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:23.709910 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:23.709910 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:23.709910 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:15:23.709910 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:15:23.709910 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:15:23.709910 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:15:23.709910 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:15:23.709910 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:15:23.709910 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:15:23.709910 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:23.709910 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 393/795] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Aug 23 04:15:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:23.721558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051aba25-3230-43f8-8203-31ee3ba46b68 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:23.768865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:23.813953 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:23.814924 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f7b7d2-b75b-4d98-bcac-304f597550a3 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:23.840108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051aba25-3230-43f8-8203-31ee3ba46b68 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:23.863821 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ac8f1-8f86-4ba3-a887-2512cf1e5b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:23.864567 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ac8f1-8f86-4ba3-a887-2512cf1e5b34 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:23.864932 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ac8f1-8f86-4ba3-a887-2512cf1e5b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:23.865545 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ac8f1-8f86-4ba3-a887-2512cf1e5b34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:23.875126 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700ac8f1-8f86-4ba3-a887-2512cf1e5b34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:23.877269 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700ac8f1-8f86-4ba3-a887-2512cf1e5b34 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:23.883001 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b2bdf-f1fd-440e-b350-9301d555316d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:23.883800 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b2bdf-f1fd-440e-b350-9301d555316d None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:23.884127 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b2bdf-f1fd-440e-b350-9301d555316d None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:23.884406 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b2bdf-f1fd-440e-b350-9301d555316d None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:23.885138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9b2bdf-f1fd-440e-b350-9301d555316d None admin] RBAC: Adding query filter params (name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:23.885446 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9b2bdf-f1fd-440e-b350-9301d555316d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:23.886856 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9b2bdf-f1fd-440e-b350-9301d555316d None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:23.893975 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-700ac8f1-8f86-4ba3-a887-2512cf1e5b34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:23.893975 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:23.893975 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:23.893975 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:23.893975 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:23.893975 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:23.893975 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:23.893975 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:23.893975 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:23.893975 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:23.893975 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:23.893975 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:23.893975 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:15:23.893975 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:15:23.893975 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:15:23.894857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:15:23.894857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:23.894857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:23.894857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:23.894857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:23.894857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:23.894857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:23.894857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:23.894857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:23.894857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:23.894857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:23.894857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:23.894857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:23.894857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:23.894857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:23.894857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:23.894857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:23.895713 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:15:23.895713 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:15:23.895713 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:15:23.895713 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:15:23.895713 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:15:23.895713 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:15:23.895713 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:23.895713 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:23.895713 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 403/796] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:15:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:23.895713 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ac8f1-8f86-4ba3-a887-2512cf1e5b34 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:23.901866 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 394/797] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Aug 23 04:15:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:23.902773 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b2bdf-f1fd-440e-b350-9301d555316d None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:24.012920 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b2bdf-f1fd-440e-b350-9301d555316d None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:24.021638 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ac8f1-8f86-4ba3-a887-2512cf1e5b34 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:24.051505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ebec9-51da-4fd4-bac6-ad98c5eefb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:24.052256 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ebec9-51da-4fd4-bac6-ad98c5eefb7d None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:24.053110 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ebec9-51da-4fd4-bac6-ad98c5eefb7d None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:24.053407 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ebec9-51da-4fd4-bac6-ad98c5eefb7d None admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:24.054150 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7ebec9-51da-4fd4-bac6-ad98c5eefb7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:24.054615 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:24.055712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7ebec9-51da-4fd4-bac6-ad98c5eefb7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:24.057626 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7ebec9-51da-4fd4-bac6-ad98c5eefb7d None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:24.061024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e090c0-c6aa-4767-bc8a-cc0d92bc678a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:24.061792 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e090c0-c6aa-4767-bc8a-cc0d92bc678a None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:24.062171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e090c0-c6aa-4767-bc8a-cc0d92bc678a None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:24.062542 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e090c0-c6aa-4767-bc8a-cc0d92bc678a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:24.073974 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e090c0-c6aa-4767-bc8a-cc0d92bc678a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:15:24.074726 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 395/798] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:15:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:24.075709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ebec9-51da-4fd4-bac6-ad98c5eefb7d None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:24.078512 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:24.079488 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-40e090c0-c6aa-4767-bc8a-cc0d92bc678a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:15:24.079488 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:24.079488 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:24.079488 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:24.079488 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:24.079488 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:24.079488 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:24.079488 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:24.079488 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:24.079488 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:24.079488 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:24.079488 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:24.079488 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:15:24.079488 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:15:24.079488 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:15:24.079488 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:15:24.082069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:15:24.082069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:15:24.082069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:15:24.082069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:15:24.082069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:24.082069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:24.082069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:24.082069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:24.082069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:24.082069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:24.082069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:24.082069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:24.082069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:24.082069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:24.082069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:24.082069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:24.082971 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:24.082971 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:24.082971 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:24.082971 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:24.082971 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:15:24.082971 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:15:24.082971 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:15:24.082971 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:15:24.082971 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:15:24.082971 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:15:24.082971 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:15:24.082971 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:24.082971 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 404/799] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:15:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:24.082971 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e090c0-c6aa-4767-bc8a-cc0d92bc678a None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:24.193843 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ebec9-51da-4fd4-bac6-ad98c5eefb7d None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:24.197529 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e090c0-c6aa-4767-bc8a-cc0d92bc678a None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:24.231556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a14f9c-29c9-4c6b-9f99-6e38c1302cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:24.232575 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a14f9c-29c9-4c6b-9f99-6e38c1302cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:24.233029 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a14f9c-29c9-4c6b-9f99-6e38c1302cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:24.233571 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a14f9c-29c9-4c6b-9f99-6e38c1302cb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:24.234044 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9dfb91-517b-4d43-af54-a575e710fc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:24.234793 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9dfb91-517b-4d43-af54-a575e710fc4a None admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:24.235145 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9dfb91-517b-4d43-af54-a575e710fc4a None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:24.235489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9dfb91-517b-4d43-af54-a575e710fc4a None admin] PATH_INFO: `/v3/projects/9f4373dfb5f9425eb73669e24d18a08f/users/3f1418fd211a448da1a50175dce13b32/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:24.237693 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9dfb91-517b-4d43-af54-a575e710fc4a None admin] RBAC: Authorizing `identity:create_grant(project_id=9f4373dfb5f9425eb73669e24d18a08f, user_id=3f1418fd211a448da1a50175dce13b32, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:24.239264 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9dfb91-517b-4d43-af54-a575e710fc4a None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:24.246844 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a14f9c-29c9-4c6b-9f99-6e38c1302cb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:24.248610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a14f9c-29c9-4c6b-9f99-6e38c1302cb8 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:24.255289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 405/800] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Aug 23 04:15:24 2022] PUT /identity/v3/projects/9f4373dfb5f9425eb73669e24d18a08f/users/3f1418fd211a448da1a50175dce13b32/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:15:24.256545 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9dfb91-517b-4d43-af54-a575e710fc4a None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:24.266100 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-32a14f9c-29c9-4c6b-9f99-6e38c1302cb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:24.266100 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:24.266100 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:24.266100 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:24.266100 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:24.266100 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:24.266100 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:24.266100 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:24.266100 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:24.266100 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:24.266100 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:24.266100 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:24.266100 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:24.266100 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:24.266100 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:24.266100 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:24.267084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:24.267084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:24.267084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:24.267084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:24.267084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:24.267084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:24.267084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:24.267084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:24.267084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:24.267084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:24.267084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:24.267084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:24.267084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:24.267084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:24.267084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:24.267084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:24.267084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:24.267988 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:24.267988 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:24.267988 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:24.270230 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 396/801] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:24.277240 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a14f9c-29c9-4c6b-9f99-6e38c1302cb8 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:24.320250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:24.384081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9dfb91-517b-4d43-af54-a575e710fc4a None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:24.394101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:24.423144 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f312c6-c0e3-4a83-9ba7-6819a53c189d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:24.423782 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f312c6-c0e3-4a83-9ba7-6819a53c189d None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:24.424095 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f312c6-c0e3-4a83-9ba7-6819a53c189d None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:24.424368 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f312c6-c0e3-4a83-9ba7-6819a53c189d None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:24.425048 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f312c6-c0e3-4a83-9ba7-6819a53c189d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:24.425353 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f312c6-c0e3-4a83-9ba7-6819a53c189d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:24.426800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f312c6-c0e3-4a83-9ba7-6819a53c189d None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:24.444470 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 406/802] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:15:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:24.446522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f312c6-c0e3-4a83-9ba7-6819a53c189d None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:24.495391 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a14f9c-29c9-4c6b-9f99-6e38c1302cb8 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:24.565519 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1cbb39-06ba-4503-82ba-280fe981a3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:24.566055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1cbb39-06ba-4503-82ba-280fe981a3d7 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:24.566335 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1cbb39-06ba-4503-82ba-280fe981a3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:24.566608 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1cbb39-06ba-4503-82ba-280fe981a3d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:24.567257 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1cbb39-06ba-4503-82ba-280fe981a3d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:24.567617 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1cbb39-06ba-4503-82ba-280fe981a3d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:24.570093 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1cbb39-06ba-4503-82ba-280fe981a3d7 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:24.598729 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 397/803] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:24.604921 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1cbb39-06ba-4503-82ba-280fe981a3d7 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:24.644269 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f312c6-c0e3-4a83-9ba7-6819a53c189d None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:24.672286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:24.711090 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e555d0-26b7-43df-aa76-46621d3597a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:24.711739 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e555d0-26b7-43df-aa76-46621d3597a5 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:24.712041 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e555d0-26b7-43df-aa76-46621d3597a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:24.712309 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e555d0-26b7-43df-aa76-46621d3597a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:24.737207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e555d0-26b7-43df-aa76-46621d3597a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:24.738681 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e555d0-26b7-43df-aa76-46621d3597a5 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:24.765889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-c1e555d0-26b7-43df-aa76-46621d3597a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:24.765889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:24.765889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:24.765889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:24.765889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:24.765889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:24.765889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:24.765889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:24.765889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:24.765889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:24.765889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:24.765889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:24.765889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:24.765889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:24.765889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:24.765889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:24.766877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:24.766877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:24.766877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:24.766877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:24.766877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:24.766877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:24.766877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:24.766877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:24.766877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:24.766877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:24.766877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:24.766877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:24.766877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:24.766877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:24.766877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:24.766877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:24.766877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:24.767753 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:24.767753 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:24.767753 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:24.772620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 407/804] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:24.775659 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e555d0-26b7-43df-aa76-46621d3597a5 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:24.787245 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1cbb39-06ba-4503-82ba-280fe981a3d7 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:24.825728 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:24.869966 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3f53a-fa7c-4eec-b375-d8ba32087707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:24.870638 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3f53a-fa7c-4eec-b375-d8ba32087707 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:24.870917 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3f53a-fa7c-4eec-b375-d8ba32087707 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:24.871179 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3f53a-fa7c-4eec-b375-d8ba32087707 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:24.886375 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d3f53a-fa7c-4eec-b375-d8ba32087707 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:24.887842 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d3f53a-fa7c-4eec-b375-d8ba32087707 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:24.900485 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-72d3f53a-fa7c-4eec-b375-d8ba32087707 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 04:15:24.900485 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:24.900485 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:24.900485 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:24.900485 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:24.900485 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:24.900485 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:24.900485 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:24.900485 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:24.900485 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:24.900485 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:24.900485 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:24.900485 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:15:24.900485 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:15:24.900485 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:15:24.900485 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:15:24.901419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:24.901419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:24.901419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:15:24.901419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:24.901419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:15:24.901419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:24.901419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:24.901419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:24.901419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:24.901419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:24.901419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:24.901419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:24.901419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:24.901419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:24.901419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:24.901419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:24.901419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:24.901419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:24.902331 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:24.902331 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:24.902331 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:15:24.902331 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:15:24.902331 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:15:24.902331 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:15:24.902331 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:15:24.902331 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:15:24.902331 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 04:15:24.902331 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:24.902331 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 398/805] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Aug 23 04:15:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:24.903466 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe799a-709c-4625-90ea-40f883ace500 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:24.903466 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe799a-709c-4625-90ea-40f883ace500 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:24.903631 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe799a-709c-4625-90ea-40f883ace500 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:24.908993 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 399/806] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:24.908993 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe799a-709c-4625-90ea-40f883ace500 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:24.930236 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e555d0-26b7-43df-aa76-46621d3597a5 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:24.977213 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34814cb7-be9a-4d37-ac80-f1c87e2bf16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:24.977828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34814cb7-be9a-4d37-ac80-f1c87e2bf16d None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:24.978102 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34814cb7-be9a-4d37-ac80-f1c87e2bf16d None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:24.978372 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34814cb7-be9a-4d37-ac80-f1c87e2bf16d None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:24.979279 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34814cb7-be9a-4d37-ac80-f1c87e2bf16d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:24.979581 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34814cb7-be9a-4d37-ac80-f1c87e2bf16d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:24.981256 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34814cb7-be9a-4d37-ac80-f1c87e2bf16d None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:24.997658 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 408/807] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:24.999373 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b1d0a-5980-4882-a58d-b0daf1256937 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:24.999660 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b1d0a-5980-4882-a58d-b0daf1256937 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:24.999949 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b1d0a-5980-4882-a58d-b0daf1256937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:25.024064 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-489b1d0a-5980-4882-a58d-b0daf1256937 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:25.025258 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-489b1d0a-5980-4882-a58d-b0daf1256937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:25.025525 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe799a-709c-4625-90ea-40f883ace500 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:25.053369 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:25.065572 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9dcbdb-041f-4382-95e3-b8b5ee7a1b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:25.066903 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9dcbdb-041f-4382-95e3-b8b5ee7a1b66 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:25.067382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9dcbdb-041f-4382-95e3-b8b5ee7a1b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:25.067981 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9dcbdb-041f-4382-95e3-b8b5ee7a1b66 None admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:25.069123 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9dcbdb-041f-4382-95e3-b8b5ee7a1b66 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:25.069741 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9dcbdb-041f-4382-95e3-b8b5ee7a1b66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:25.071575 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9dcbdb-041f-4382-95e3-b8b5ee7a1b66 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:25.091459 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 400/808] 10.222.0.18 () {56 vars in 1149 bytes} [Tue Aug 23 04:15:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:25.092689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9dcbdb-041f-4382-95e3-b8b5ee7a1b66 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:25.119939 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 409/809] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:15:25.119939 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b1d0a-5980-4882-a58d-b0daf1256937 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:25.139629 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:25.238064 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9dcbdb-041f-4382-95e3-b8b5ee7a1b66 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:25.243035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b1d0a-5980-4882-a58d-b0daf1256937 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:25.277565 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d793f0-ad84-4448-8cc4-3e0711b653a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:25.278257 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d793f0-ad84-4448-8cc4-3e0711b653a2 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:25.278536 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d793f0-ad84-4448-8cc4-3e0711b653a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:25.278848 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d793f0-ad84-4448-8cc4-3e0711b653a2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:25.286028 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90427a5-8e3d-4234-b2a4-d417b6cfe189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:25.286265 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d793f0-ad84-4448-8cc4-3e0711b653a2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:25.286720 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90427a5-8e3d-4234-b2a4-d417b6cfe189 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:25.287000 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90427a5-8e3d-4234-b2a4-d417b6cfe189 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:25.287269 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90427a5-8e3d-4234-b2a4-d417b6cfe189 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:25.288566 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d793f0-ad84-4448-8cc4-3e0711b653a2 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:25.299183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90427a5-8e3d-4234-b2a4-d417b6cfe189 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:25.307993 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90427a5-8e3d-4234-b2a4-d417b6cfe189 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:25.308172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-c4d793f0-ad84-4448-8cc4-3e0711b653a2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 04:15:25.308172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:25.308172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:25.308172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:25.308172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:25.308172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:25.308172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:25.308172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:25.308172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:25.308172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:25.308172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:25.308172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:25.308172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:15:25.308172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:15:25.308172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:15:25.308172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:15:25.309025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:25.309025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:25.309025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:15:25.309025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:25.309025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:15:25.309025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:25.309025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:25.309025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:25.309025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:25.309025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:25.309025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:25.309025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:25.309025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:25.309025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:25.309025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:25.309025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:25.309025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:25.309025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:25.309937 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:25.309937 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:25.309937 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:15:25.309937 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:15:25.309937 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:15:25.309937 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:15:25.309937 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:15:25.309937 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:15:25.309937 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 04:15:25.309937 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:25.309937 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 401/810] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Aug 23 04:15:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:25.309937 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecbdb5-cb55-4948-870e-b4fb105a2d71 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:25.309937 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecbdb5-cb55-4948-870e-b4fb105a2d71 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:25.310615 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecbdb5-cb55-4948-870e-b4fb105a2d71 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:25.311213 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 402/811] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:25.315303 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecbdb5-cb55-4948-870e-b4fb105a2d71 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:25.315512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-a90427a5-8e3d-4234-b2a4-d417b6cfe189 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:25.315512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:25.315512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:25.315512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:25.315512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:25.315512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:25.315512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:25.315512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:25.315512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:25.315512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:25.315512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:25.315512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:25.315512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:25.315512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:25.315512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:25.315512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:25.316427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:25.316427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:25.316427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:25.316427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:25.316427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:25.316427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:25.316427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:25.316427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:25.316427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:25.316427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:25.316427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:25.316427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:25.316427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:25.316427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:25.316427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:25.316427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:25.316427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:25.319263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:25.319263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:25.319263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:25.319263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 410/812] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:25.319263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b679b-b5ee-45fc-8443-972e9d44361f None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:25.319263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b679b-b5ee-45fc-8443-972e9d44361f None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:25.319263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b679b-b5ee-45fc-8443-972e9d44361f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:25.351869 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-375b679b-b5ee-45fc-8443-972e9d44361f None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:25.353915 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-375b679b-b5ee-45fc-8443-972e9d44361f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:25.357268 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:25.436653 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:25.447252 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecbdb5-cb55-4948-870e-b4fb105a2d71 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:25.455939 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 411/813] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 04:15:25.455939 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b679b-b5ee-45fc-8443-972e9d44361f None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:25.494341 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d5ec9-99d1-4b39-bdf4-73fd63518ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:25.494947 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d5ec9-99d1-4b39-bdf4-73fd63518ce0 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:25.495219 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d5ec9-99d1-4b39-bdf4-73fd63518ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:25.495490 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d5ec9-99d1-4b39-bdf4-73fd63518ce0 None admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:25.496244 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d5ec9-99d1-4b39-bdf4-73fd63518ce0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:25.496542 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d5ec9-99d1-4b39-bdf4-73fd63518ce0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:25.497980 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d5ec9-99d1-4b39-bdf4-73fd63518ce0 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:25.517226 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 403/814] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Aug 23 04:15:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:25.518059 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d5ec9-99d1-4b39-bdf4-73fd63518ce0 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:25.590913 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b679b-b5ee-45fc-8443-972e9d44361f None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:25.623507 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d5ec9-99d1-4b39-bdf4-73fd63518ce0 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:25.636939 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d633c34a-efdd-4c0e-8a77-7720098751ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:25.637807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633c34a-efdd-4c0e-8a77-7720098751ca None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:25.638552 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633c34a-efdd-4c0e-8a77-7720098751ca None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:25.638834 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633c34a-efdd-4c0e-8a77-7720098751ca None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:25.639760 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d633c34a-efdd-4c0e-8a77-7720098751ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:25.640094 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d633c34a-efdd-4c0e-8a77-7720098751ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:25.642088 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d633c34a-efdd-4c0e-8a77-7720098751ca None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:25.650846 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:25.659156 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 412/815] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:25.660212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d633c34a-efdd-4c0e-8a77-7720098751ca None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:25.665651 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62996a49-50ec-437d-a16f-7306f7f8bc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:25.666299 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62996a49-50ec-437d-a16f-7306f7f8bc38 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:25.666627 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62996a49-50ec-437d-a16f-7306f7f8bc38 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:25.666896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62996a49-50ec-437d-a16f-7306f7f8bc38 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:25.667648 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62996a49-50ec-437d-a16f-7306f7f8bc38 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:25.669102 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62996a49-50ec-437d-a16f-7306f7f8bc38 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:25.679385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-62996a49-50ec-437d-a16f-7306f7f8bc38 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 04:15:25.679385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:25.679385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 04:15:25.679385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 04:15:25.679385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 04:15:25.679385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 04:15:25.679385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 04:15:25.679385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:25.679385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 04:15:25.679385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:25.679385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:15:25.679385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:25.679385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:25.679385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:25.679385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:25.679385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:25.679385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:25.680630 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:25.680630 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:25.680630 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:25.680630 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:25.680630 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:25.680630 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:25.680630 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 04:15:25.680630 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 04:15:25.680630 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 04:15:25.680630 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 04:15:25.680630 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:25.680630 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:25.680630 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:25.680630 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:25.680630 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:25.680630 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:25.681481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:25.681481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:25.681481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:25.681481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:25.681481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:25.681481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:25.681481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:25.681481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:25.681481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:25.681481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:25.681481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 04:15:25.681481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:25.681481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 04:15:25.681481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:25.681481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 404/816] 10.222.0.18 () {56 vars in 1129 bytes} [Tue Aug 23 04:15:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:25.682266 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62996a49-50ec-437d-a16f-7306f7f8bc38 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:25.708588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:25.775989 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d633c34a-efdd-4c0e-8a77-7720098751ca None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:25.802015 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62996a49-50ec-437d-a16f-7306f7f8bc38 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:25.812485 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af49081-69ec-48e7-bbb1-a12f282667c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:25.813115 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af49081-69ec-48e7-bbb1-a12f282667c5 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:25.813701 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af49081-69ec-48e7-bbb1-a12f282667c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:25.813989 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af49081-69ec-48e7-bbb1-a12f282667c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:25.839965 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3076f-9b55-458b-9106-6c576d1943af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:25.840601 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3076f-9b55-458b-9106-6c576d1943af None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:25.840873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3076f-9b55-458b-9106-6c576d1943af None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:25.841136 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3076f-9b55-458b-9106-6c576d1943af None admin] PATH_INFO: `/v3/projects/service` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:25.848079 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd3076f-9b55-458b-9106-6c576d1943af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:25.849472 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd3076f-9b55-458b-9106-6c576d1943af None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:25.861234 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-5dd3076f-9b55-458b-9106-6c576d1943af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:25.861234 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:25.861234 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:25.861234 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:25.861234 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:25.861234 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:25.861234 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:25.861234 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:25.861234 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:25.861234 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:25.861234 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:25.861234 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:25.861234 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:15:25.861234 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:15:25.861234 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:15:25.862159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:15:25.862159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:25.862159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:25.862159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:25.862159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:25.862159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:25.862159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:25.862159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:25.862159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:25.862159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:25.862159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:25.862159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:25.862159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:25.862159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:25.862159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:25.862159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:25.862159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:25.863012 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:15:25.863012 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:15:25.863012 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:15:25.863012 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:15:25.863012 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:15:25.863012 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:15:25.863012 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:25.863012 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:25.867270 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 405/817] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:15:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:25.868141 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3076f-9b55-458b-9106-6c576d1943af None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:25.889087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:25.894101 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af49081-69ec-48e7-bbb1-a12f282667c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:25.897824 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af49081-69ec-48e7-bbb1-a12f282667c5 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:25.948415 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:25.977178 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3076f-9b55-458b-9106-6c576d1943af None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:26.004924 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 413/818] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:15:25 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 04:15:26.005829 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af49081-69ec-48e7-bbb1-a12f282667c5 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:26.018783 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269b5f90-8186-4569-89f6-74b6aa918094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:26.019411 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b5f90-8186-4569-89f6-74b6aa918094 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:26.019689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b5f90-8186-4569-89f6-74b6aa918094 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:26.019986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b5f90-8186-4569-89f6-74b6aa918094 None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:26.020710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269b5f90-8186-4569-89f6-74b6aa918094 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:26.021084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269b5f90-8186-4569-89f6-74b6aa918094 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:26.023176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269b5f90-8186-4569-89f6-74b6aa918094 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:26.028657 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 406/819] 10.222.0.18 () {56 vars in 1124 bytes} [Tue Aug 23 04:15:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:26.029460 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269b5f90-8186-4569-89f6-74b6aa918094 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:26.111066 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:26.116906 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af49081-69ec-48e7-bbb1-a12f282667c5 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:26.140190 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269b5f90-8186-4569-89f6-74b6aa918094 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:26.155203 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5033d-bc7f-48e7-9d19-d54ad0ca51ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:26.155781 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5033d-bc7f-48e7-9d19-d54ad0ca51ff None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:26.156101 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5033d-bc7f-48e7-9d19-d54ad0ca51ff None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:26.156449 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5033d-bc7f-48e7-9d19-d54ad0ca51ff None admin] PATH_INFO: `/v3/services/network` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:26.156956 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5033d-bc7f-48e7-9d19-d54ad0ca51ff None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:26.158363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5033d-bc7f-48e7-9d19-d54ad0ca51ff None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:26.167359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-bee5033d-bc7f-48e7-9d19-d54ad0ca51ff None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 04:15:26.167359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:26.167359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 04:15:26.167359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 04:15:26.167359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 04:15:26.167359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 04:15:26.167359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 04:15:26.167359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:26.167359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 04:15:26.167359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:26.167359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:15:26.167359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:26.167359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:26.167359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:26.167359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:26.167359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:26.167359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:26.168430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:26.168430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:26.168430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:26.168430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:26.168430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:26.168430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:26.168430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 04:15:26.168430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 04:15:26.168430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 04:15:26.168430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 04:15:26.168430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:26.168430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:26.168430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:26.168430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:26.168430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:26.168430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:26.169239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:26.169239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:26.169239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:26.169239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:26.169239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:26.169239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:26.169239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:26.169239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:26.169239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:26.169239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:26.169239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 04:15:26.169239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:26.169239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 04:15:26.169239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:26.169239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 414/820] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:15:26 2022] GET /identity/v3/services/network => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:26.170002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5033d-bc7f-48e7-9d19-d54ad0ca51ff None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:26.182909 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d4b1c-a7ee-4c7a-a741-9ff1eee9c75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:26.183547 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d4b1c-a7ee-4c7a-a741-9ff1eee9c75e None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:26.183826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d4b1c-a7ee-4c7a-a741-9ff1eee9c75e None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:26.184126 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d4b1c-a7ee-4c7a-a741-9ff1eee9c75e None admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:26.184823 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d4b1c-a7ee-4c7a-a741-9ff1eee9c75e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:26.185129 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d4b1c-a7ee-4c7a-a741-9ff1eee9c75e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:26.186851 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d4b1c-a7ee-4c7a-a741-9ff1eee9c75e None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:26.196486 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:26.203078 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 407/821] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:15:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:26.203905 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d4b1c-a7ee-4c7a-a741-9ff1eee9c75e None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:26.277894 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5033d-bc7f-48e7-9d19-d54ad0ca51ff None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:26.312114 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1757dc7-30fd-419c-b940-7578878abafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:26.312743 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1757dc7-30fd-419c-b940-7578878abafd None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:26.313022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1757dc7-30fd-419c-b940-7578878abafd None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:26.313297 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1757dc7-30fd-419c-b940-7578878abafd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:26.315661 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d4b1c-a7ee-4c7a-a741-9ff1eee9c75e None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:26.323978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1757dc7-30fd-419c-b940-7578878abafd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:26.325434 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1757dc7-30fd-419c-b940-7578878abafd None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:26.340049 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-f1757dc7-30fd-419c-b940-7578878abafd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:26.340049 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:26.340049 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:26.340049 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:26.340049 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:26.340049 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:26.340049 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:26.340049 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:26.340049 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:26.340049 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:26.340049 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:26.340049 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:26.340049 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:26.340049 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:26.340049 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:26.340049 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:26.340996 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:26.340996 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:26.340996 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:26.340996 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:26.340996 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:26.340996 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:26.340996 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:26.340996 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:26.340996 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:26.340996 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:26.340996 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:26.340996 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:26.340996 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:26.340996 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:26.340996 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:26.340996 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:26.340996 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:26.341860 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:26.341860 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:26.341860 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:26.341860 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 415/822] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:26.342091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1757dc7-30fd-419c-b940-7578878abafd None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:26.353585 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f61b6f-357b-4adf-b295-cef7c5b5e881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:26.354201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f61b6f-357b-4adf-b295-cef7c5b5e881 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:26.354477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f61b6f-357b-4adf-b295-cef7c5b5e881 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:26.354744 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f61b6f-357b-4adf-b295-cef7c5b5e881 None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:26.355374 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f61b6f-357b-4adf-b295-cef7c5b5e881 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:26.355669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f61b6f-357b-4adf-b295-cef7c5b5e881 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:26.357068 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f61b6f-357b-4adf-b295-cef7c5b5e881 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:26.360376 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:26.363188 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 408/823] 10.222.0.18 () {56 vars in 1124 bytes} [Tue Aug 23 04:15:26 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:26.363967 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f61b6f-357b-4adf-b295-cef7c5b5e881 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:26.429116 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:26.444349 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1757dc7-30fd-419c-b940-7578878abafd None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:26.467843 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f61b6f-357b-4adf-b295-cef7c5b5e881 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:26.483794 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554de42d-6942-4cf9-9c77-dca952cd0307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:26.484522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554de42d-6942-4cf9-9c77-dca952cd0307 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:26.484802 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554de42d-6942-4cf9-9c77-dca952cd0307 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:26.485075 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554de42d-6942-4cf9-9c77-dca952cd0307 None admin] PATH_INFO: `/v3/services` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:26.485715 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554de42d-6942-4cf9-9c77-dca952cd0307 None admin] RBAC: Adding query filter params (name=network) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:26.486016 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554de42d-6942-4cf9-9c77-dca952cd0307 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:26.487392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554de42d-6942-4cf9-9c77-dca952cd0307 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:26.497064 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 416/824] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Aug 23 04:15:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:26.497064 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0658382-a4d0-42b7-ad34-50190c9a66cc None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:26.497064 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0658382-a4d0-42b7-ad34-50190c9a66cc None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:26.497064 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0658382-a4d0-42b7-ad34-50190c9a66cc None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:26.497064 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 417/825] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:26.497064 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0658382-a4d0-42b7-ad34-50190c9a66cc None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:26.509443 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce19bd-4768-4341-b839-028614383cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:26.510188 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce19bd-4768-4341-b839-028614383cc1 None admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:26.510529 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce19bd-4768-4341-b839-028614383cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:26.510795 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce19bd-4768-4341-b839-028614383cc1 None admin] PATH_INFO: `/v3/projects/9f4373dfb5f9425eb73669e24d18a08f/users/d3538271ebfd41aabe9f3da85feb7bca/roles/bdf71ef6ab10430e8c356cd4d952ff5c` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:26.526046 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce19bd-4768-4341-b839-028614383cc1 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f4373dfb5f9425eb73669e24d18a08f, user_id=d3538271ebfd41aabe9f3da85feb7bca, role_id=bdf71ef6ab10430e8c356cd4d952ff5c)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:26.527488 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce19bd-4768-4341-b839-028614383cc1 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:26.540494 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 409/826] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Aug 23 04:15:26 2022] PUT /identity/v3/projects/9f4373dfb5f9425eb73669e24d18a08f/users/d3538271ebfd41aabe9f3da85feb7bca/roles/bdf71ef6ab10430e8c356cd4d952ff5c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:15:26.541430 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce19bd-4768-4341-b839-028614383cc1 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:26.596569 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:26.625101 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0658382-a4d0-42b7-ad34-50190c9a66cc None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:26.662296 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce19bd-4768-4341-b839-028614383cc1 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:26.664848 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cea8fe-ab36-4004-8bfa-866d4dc572a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:26.665511 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cea8fe-ab36-4004-8bfa-866d4dc572a9 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:26.665786 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cea8fe-ab36-4004-8bfa-866d4dc572a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:26.666061 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cea8fe-ab36-4004-8bfa-866d4dc572a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:26.666729 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cea8fe-ab36-4004-8bfa-866d4dc572a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:26.667032 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cea8fe-ab36-4004-8bfa-866d4dc572a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:26.668504 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cea8fe-ab36-4004-8bfa-866d4dc572a9 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:26.680969 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:26.684939 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 418/827] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:26.684939 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cea8fe-ab36-4004-8bfa-866d4dc572a9 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:26.712116 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc687d02-62d3-4775-8034-d7ad7593bff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:26.712738 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc687d02-62d3-4775-8034-d7ad7593bff0 None admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:26.713007 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc687d02-62d3-4775-8034-d7ad7593bff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:26.713637 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc687d02-62d3-4775-8034-d7ad7593bff0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:26.714294 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc687d02-62d3-4775-8034-d7ad7593bff0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:26.715670 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc687d02-62d3-4775-8034-d7ad7593bff0 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:26.739058 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 410/828] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:15:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:15:26.740405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829ae26-2b12-44f5-a5dd-8b16bfea3335 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:26.740660 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829ae26-2b12-44f5-a5dd-8b16bfea3335 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:26.740931 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829ae26-2b12-44f5-a5dd-8b16bfea3335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:26.772143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-1829ae26-2b12-44f5-a5dd-8b16bfea3335 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:26.773997 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-1829ae26-2b12-44f5-a5dd-8b16bfea3335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:26.821329 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cea8fe-ab36-4004-8bfa-866d4dc572a9 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:26.866226 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 411/829] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:15:26.867271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829ae26-2b12-44f5-a5dd-8b16bfea3335 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:26.878059 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c71809-e873-499a-83e7-194d252c3480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:26.879109 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c71809-e873-499a-83e7-194d252c3480 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:26.879609 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c71809-e873-499a-83e7-194d252c3480 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:26.880253 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c71809-e873-499a-83e7-194d252c3480 None admin] PATH_INFO: `/v3/services` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:26.881217 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c71809-e873-499a-83e7-194d252c3480 None admin] RBAC: Adding query filter params (type=network) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:26.881836 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c71809-e873-499a-83e7-194d252c3480 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:26.883424 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c71809-e873-499a-83e7-194d252c3480 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:26.885353 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:26.890568 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 419/830] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Aug 23 04:15:26 2022] GET /identity/v3/services?type=network => generated 350 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:26.896403 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c71809-e873-499a-83e7-194d252c3480 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:26.995251 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:27.002192 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829ae26-2b12-44f5-a5dd-8b16bfea3335 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:27.031845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c71809-e873-499a-83e7-194d252c3480 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:27.042260 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1115a267-f59d-41a9-ad93-28f1f7188cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:27.043060 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115a267-f59d-41a9-ad93-28f1f7188cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:27.043440 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115a267-f59d-41a9-ad93-28f1f7188cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:27.043804 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115a267-f59d-41a9-ad93-28f1f7188cf7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:27.050835 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1115a267-f59d-41a9-ad93-28f1f7188cf7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:27.052455 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1115a267-f59d-41a9-ad93-28f1f7188cf7 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:27.063896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-1115a267-f59d-41a9-ad93-28f1f7188cf7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:27.063896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:27.063896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:27.063896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:27.063896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:27.063896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:27.063896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:27.063896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:27.063896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:27.063896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:27.063896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:27.063896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:27.063896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:15:27.063896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:15:27.063896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:15:27.064932 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:15:27.064932 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:27.064932 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:27.064932 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:27.064932 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:27.064932 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:27.064932 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:27.064932 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:27.064932 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:27.064932 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:27.064932 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:27.064932 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:27.064932 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:27.064932 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:27.064932 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:27.064932 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:27.064932 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:27.066002 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:15:27.066002 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:15:27.066002 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:15:27.066002 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:15:27.066002 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:15:27.066002 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:15:27.066002 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:27.066002 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:27.068389 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 412/831] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:15:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:27.071030 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b92e7-9062-476b-9b81-e58bcf317fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:27.071658 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b92e7-9062-476b-9b81-e58bcf317fbe None admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:27.072869 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b92e7-9062-476b-9b81-e58bcf317fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:27.072869 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b92e7-9062-476b-9b81-e58bcf317fbe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:27.073057 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1b92e7-9062-476b-9b81-e58bcf317fbe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:27.073295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1115a267-f59d-41a9-ad93-28f1f7188cf7 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:27.074424 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1b92e7-9062-476b-9b81-e58bcf317fbe None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:27.114057 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 420/832] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:15:26 2022] POST /identity/v3/endpoints => generated 337 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:15:27.164542 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b92e7-9062-476b-9b81-e58bcf317fbe None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:27.229707 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1115a267-f59d-41a9-ad93-28f1f7188cf7 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:27.260863 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:27.323650 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5a654-076a-48b1-9ef1-391e5a0cfa77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:27.324392 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5a654-076a-48b1-9ef1-391e5a0cfa77 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:27.324669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5a654-076a-48b1-9ef1-391e5a0cfa77 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:27.324933 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5a654-076a-48b1-9ef1-391e5a0cfa77 None admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:27.325735 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe5a654-076a-48b1-9ef1-391e5a0cfa77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:27.326041 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe5a654-076a-48b1-9ef1-391e5a0cfa77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:27.327491 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe5a654-076a-48b1-9ef1-391e5a0cfa77 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:27.344339 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 413/833] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:15:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:27.349766 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b92e7-9062-476b-9b81-e58bcf317fbe None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:27.350427 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5a654-076a-48b1-9ef1-391e5a0cfa77 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:27.366528 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:27.392090 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6f2b0-a5bf-44ac-8603-76324290bd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:27.392826 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6f2b0-a5bf-44ac-8603-76324290bd92 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:27.393113 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6f2b0-a5bf-44ac-8603-76324290bd92 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:27.393449 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6f2b0-a5bf-44ac-8603-76324290bd92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:27.405263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6f2b0-a5bf-44ac-8603-76324290bd92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:15:27.412744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-1fc6f2b0-a5bf-44ac-8603-76324290bd92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:15:27.412744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:27.412744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:27.412744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:27.412744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:27.412744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:27.412744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:27.412744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:27.412744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:27.412744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:27.412744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:27.412744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:27.412744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:15:27.412744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:15:27.412744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:15:27.412744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:15:27.413685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:15:27.413685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:15:27.413685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:15:27.413685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:15:27.413685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:27.413685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:27.413685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:27.413685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:27.413685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:27.413685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:27.413685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:27.413685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:27.413685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:27.413685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:27.413685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:27.413685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:27.414482 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:27.414482 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:27.414482 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:27.414482 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:27.414482 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:15:27.414482 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:15:27.414482 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:15:27.414482 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:15:27.414482 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:15:27.414482 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:15:27.414482 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:15:27.414482 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:27.414482 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 421/834] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:15:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:15:27.419698 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6f2b0-a5bf-44ac-8603-76324290bd92 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:27.493892 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5a654-076a-48b1-9ef1-391e5a0cfa77 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:27.535276 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abb2ad-6737-4f7b-8241-33c3a68e1fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:27.535959 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abb2ad-6737-4f7b-8241-33c3a68e1fd1 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:27.536339 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abb2ad-6737-4f7b-8241-33c3a68e1fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:27.536866 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abb2ad-6737-4f7b-8241-33c3a68e1fd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:27.537739 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13abb2ad-6737-4f7b-8241-33c3a68e1fd1 None admin] RBAC: Adding query filter params (role.id=52b31061b72c45ccb76dd26bf31f8d3f, scope.project.id=9f4373dfb5f9425eb73669e24d18a08f, user.id=cd7b11f479a54adc8d9b8641859b07a1) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:27.538123 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13abb2ad-6737-4f7b-8241-33c3a68e1fd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:27.539750 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13abb2ad-6737-4f7b-8241-33c3a68e1fd1 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:27.546761 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 414/835] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:15:27 2022] GET /identity/v3/role_assignments?user.id=cd7b11f479a54adc8d9b8641859b07a1&scope.project.id=9f4373dfb5f9425eb73669e24d18a08f&role.id=52b31061b72c45ccb76dd26bf31f8d3f => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:27.554462 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6f2b0-a5bf-44ac-8603-76324290bd92 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:27.571969 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:27.596452 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f5361-57ad-4bc8-a1df-c6fbce255e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:27.597085 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f5361-57ad-4bc8-a1df-c6fbce255e7d None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:27.597364 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f5361-57ad-4bc8-a1df-c6fbce255e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:27.597634 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f5361-57ad-4bc8-a1df-c6fbce255e7d None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:27.598320 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f5361-57ad-4bc8-a1df-c6fbce255e7d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:27.598625 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f5361-57ad-4bc8-a1df-c6fbce255e7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:27.600595 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f5361-57ad-4bc8-a1df-c6fbce255e7d None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:27.611775 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 422/836] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:15:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:27.617192 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abb2ad-6737-4f7b-8241-33c3a68e1fd1 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:27.707989 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:27.738004 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abb2ad-6737-4f7b-8241-33c3a68e1fd1 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:27.774894 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98191fb5-9760-439b-b4e1-8ef7bfc5a036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:27.775538 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98191fb5-9760-439b-b4e1-8ef7bfc5a036 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:27.775810 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98191fb5-9760-439b-b4e1-8ef7bfc5a036 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:27.776135 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98191fb5-9760-439b-b4e1-8ef7bfc5a036 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:27.787099 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98191fb5-9760-439b-b4e1-8ef7bfc5a036 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:27.788592 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98191fb5-9760-439b-b4e1-8ef7bfc5a036 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:27.802841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-98191fb5-9760-439b-b4e1-8ef7bfc5a036 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:27.802841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:27.802841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:27.802841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:27.802841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:27.802841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:27.802841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:27.802841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:27.802841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:27.802841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:27.802841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:27.802841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:27.802841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:27.802841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:27.802841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:27.802841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:27.803794 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:27.803794 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:27.803794 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:27.803794 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:27.803794 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:27.803794 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:27.803794 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:27.803794 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:27.803794 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:27.803794 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:27.803794 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:27.803794 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:27.803794 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:27.803794 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:27.803794 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:27.803794 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:27.803794 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:27.805373 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:27.805373 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:27.805373 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:27.805373 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 415/837] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:15:27.810059 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f5361-57ad-4bc8-a1df-c6fbce255e7d None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:27.914883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f5361-57ad-4bc8-a1df-c6fbce255e7d None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:27.952081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72519103-7439-48a3-b3d2-753f591f99ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:27.953124 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72519103-7439-48a3-b3d2-753f591f99ab None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:27.953605 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72519103-7439-48a3-b3d2-753f591f99ab None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:27.954093 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72519103-7439-48a3-b3d2-753f591f99ab None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:27.954943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72519103-7439-48a3-b3d2-753f591f99ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:27.955442 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72519103-7439-48a3-b3d2-753f591f99ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:27.957042 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72519103-7439-48a3-b3d2-753f591f99ab None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:27.970339 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 423/838] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:27.976270 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98191fb5-9760-439b-b4e1-8ef7bfc5a036 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:28.078723 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98191fb5-9760-439b-b4e1-8ef7bfc5a036 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:28.105198 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:28.114857 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf186925-d865-43f1-bb25-5dba4d389e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:28.115500 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf186925-d865-43f1-bb25-5dba4d389e4a None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:28.115774 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf186925-d865-43f1-bb25-5dba4d389e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:28.116071 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf186925-d865-43f1-bb25-5dba4d389e4a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:28.124221 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf186925-d865-43f1-bb25-5dba4d389e4a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:28.125674 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf186925-d865-43f1-bb25-5dba4d389e4a None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:28.137367 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-bf186925-d865-43f1-bb25-5dba4d389e4a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 04:15:28.137367 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:28.137367 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:28.137367 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:28.137367 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:28.137367 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:28.137367 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:28.137367 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:28.137367 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:28.137367 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:28.137367 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:28.137367 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:28.137367 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:15:28.137367 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:15:28.137367 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:15:28.137367 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:15:28.138344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:28.138344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:28.138344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:15:28.138344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:28.138344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:15:28.138344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:28.138344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:28.138344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:28.138344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:28.138344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:28.138344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:28.138344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:28.138344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:28.138344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:28.138344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:28.138344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:28.138344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:28.138344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:28.140108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:28.140108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:28.140108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:15:28.140108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:15:28.140108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:15:28.140108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:15:28.140108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:15:28.140108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:15:28.140108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 04:15:28.140108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:28.140108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 416/839] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:15:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:28.144541 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72519103-7439-48a3-b3d2-753f591f99ab None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:28.185313 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:28.250316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72519103-7439-48a3-b3d2-753f591f99ab None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:28.283636 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54850be1-8b57-4b6c-a79c-9f01700ce2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:28.284272 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54850be1-8b57-4b6c-a79c-9f01700ce2a7 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:28.284568 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54850be1-8b57-4b6c-a79c-9f01700ce2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:28.284808 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54850be1-8b57-4b6c-a79c-9f01700ce2a7 None admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:28.285582 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54850be1-8b57-4b6c-a79c-9f01700ce2a7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:28.285885 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54850be1-8b57-4b6c-a79c-9f01700ce2a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:28.287353 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54850be1-8b57-4b6c-a79c-9f01700ce2a7 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:28.304972 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 424/840] 10.222.0.18 () {56 vars in 1135 bytes} [Tue Aug 23 04:15:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:28.310550 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf186925-d865-43f1-bb25-5dba4d389e4a None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:28.414461 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf186925-d865-43f1-bb25-5dba4d389e4a None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:28.449269 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b01d84-530b-4490-ba89-c4fbb5042ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:28.449881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b01d84-530b-4490-ba89-c4fbb5042ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:28.450155 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b01d84-530b-4490-ba89-c4fbb5042ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:28.450420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b01d84-530b-4490-ba89-c4fbb5042ef3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:28.507982 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:28.521821 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b01d84-530b-4490-ba89-c4fbb5042ef3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:28.523629 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b01d84-530b-4490-ba89-c4fbb5042ef3 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:28.532080 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392896b-3331-4e24-85f3-9c465d7f4b75 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:28.532366 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392896b-3331-4e24-85f3-9c465d7f4b75 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:28.532633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392896b-3331-4e24-85f3-9c465d7f4b75 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:28.534655 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 425/841] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:15:28.540664 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a1047-ce8b-4c16-9a37-6bfe948b477c None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:28.540944 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a1047-ce8b-4c16-9a37-6bfe948b477c None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:28.541212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a1047-ce8b-4c16-9a37-6bfe948b477c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:28.562769 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-7c9a1047-ce8b-4c16-9a37-6bfe948b477c None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:28.563991 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-7c9a1047-ce8b-4c16-9a37-6bfe948b477c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:28.637488 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 417/842] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:15:28 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 04:15:28.640216 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 426/843] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:15:28.642921 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b01d84-530b-4490-ba89-c4fbb5042ef3 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:28.733597 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:28.748714 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b01d84-530b-4490-ba89-c4fbb5042ef3 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:28.783885 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ef761-93bc-4e6a-b275-5fd86e88978e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:28.784541 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ef761-93bc-4e6a-b275-5fd86e88978e None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:28.784815 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ef761-93bc-4e6a-b275-5fd86e88978e None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:28.785079 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ef761-93bc-4e6a-b275-5fd86e88978e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:28.795425 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ef761-93bc-4e6a-b275-5fd86e88978e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:28.796957 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ef761-93bc-4e6a-b275-5fd86e88978e None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:28.811337 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-4f5ef761-93bc-4e6a-b275-5fd86e88978e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:28.811337 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:28.811337 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:28.811337 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:28.811337 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:28.811337 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:28.811337 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:28.811337 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:28.811337 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:28.811337 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:28.811337 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:28.811337 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:28.811337 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:28.811337 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:28.811337 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:28.811337 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:28.812284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:28.812284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:28.812284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:28.812284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:28.812284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:28.812284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:28.812284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:28.812284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:28.812284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:28.812284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:28.812284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:28.812284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:28.812284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:28.812284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:28.812284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:28.812284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:28.812284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:28.813581 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:28.813581 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:28.813581 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:28.813581 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 418/844] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:28.818476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a1047-ce8b-4c16-9a37-6bfe948b477c None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:28.845558 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:28.914204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ef761-93bc-4e6a-b275-5fd86e88978e None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:28.935940 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a1047-ce8b-4c16-9a37-6bfe948b477c None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:28.979454 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd643f1c-3df4-427e-a885-a8c1a5b198ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:28.980118 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd643f1c-3df4-427e-a885-a8c1a5b198ad None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:28.980398 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd643f1c-3df4-427e-a885-a8c1a5b198ad None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:28.980673 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd643f1c-3df4-427e-a885-a8c1a5b198ad None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:28.981436 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd643f1c-3df4-427e-a885-a8c1a5b198ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:28.981749 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd643f1c-3df4-427e-a885-a8c1a5b198ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:28.983152 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd643f1c-3df4-427e-a885-a8c1a5b198ad None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:28.996113 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 427/845] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:29.001108 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd643f1c-3df4-427e-a885-a8c1a5b198ad None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:29.042702 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ef761-93bc-4e6a-b275-5fd86e88978e None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:29.078622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b8d5d-e407-4323-9cd0-e1c89a1d25b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:29.079300 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b8d5d-e407-4323-9cd0-e1c89a1d25b0 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:29.079573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b8d5d-e407-4323-9cd0-e1c89a1d25b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:29.079853 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b8d5d-e407-4323-9cd0-e1c89a1d25b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:29.088633 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:29.093618 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:29.094936 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b8d5d-e407-4323-9cd0-e1c89a1d25b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:15:29.100352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-eb7b8d5d-e407-4323-9cd0-e1c89a1d25b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:15:29.100352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:29.100352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:29.100352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:29.100352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:29.100352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:29.100352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:29.100352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:29.100352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:29.100352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:29.100352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:29.100352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:29.100352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:15:29.100352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:15:29.100352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:15:29.100352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:15:29.101273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:15:29.101273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:15:29.101273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:15:29.101273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:15:29.101273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:29.101273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:29.101273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:29.101273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:29.101273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:29.101273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:29.101273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:29.101273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:29.101273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:29.101273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:29.101273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:29.101273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:29.102115 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:29.102115 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:29.102115 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:29.102115 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:29.102115 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:15:29.102115 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:15:29.102115 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:15:29.102115 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:15:29.102115 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:15:29.102115 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:15:29.102115 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:15:29.102115 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:29.102115 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 419/846] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Aug 23 04:15:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:29.103397 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a67d30-9cf4-4d04-bacd-1a7a175e33a4 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:29.103397 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a67d30-9cf4-4d04-bacd-1a7a175e33a4 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:29.103397 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a67d30-9cf4-4d04-bacd-1a7a175e33a4 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:29.104843 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 420/847] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:29.107770 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a67d30-9cf4-4d04-bacd-1a7a175e33a4 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:29.117106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd643f1c-3df4-427e-a885-a8c1a5b198ad None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:29.152125 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50073211-b988-427a-9c06-fca529ec75e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:29.152753 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50073211-b988-427a-9c06-fca529ec75e8 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:29.153031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50073211-b988-427a-9c06-fca529ec75e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:29.153296 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50073211-b988-427a-9c06-fca529ec75e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:29.159892 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50073211-b988-427a-9c06-fca529ec75e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:29.161340 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50073211-b988-427a-9c06-fca529ec75e8 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:29.172519 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-50073211-b988-427a-9c06-fca529ec75e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:29.172519 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:29.172519 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:29.172519 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:29.172519 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:29.172519 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:29.172519 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:29.172519 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:29.172519 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:29.172519 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:29.172519 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:29.172519 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:29.172519 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:15:29.172519 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:15:29.172519 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:15:29.173405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:15:29.173405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:29.173405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:29.173405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:29.173405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:29.173405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:29.173405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:29.173405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:29.173405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:29.173405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:29.173405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:29.173405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:29.173405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:29.173405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:29.173405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:29.173405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:29.173405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:29.174269 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:15:29.174269 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:15:29.174269 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:15:29.174269 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:15:29.174269 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:15:29.174269 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:15:29.174269 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:29.174269 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:29.178472 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 428/848] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:15:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:15:29.179731 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82e1da-a1a3-4e3b-84bf-f411b5cea350 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:29.180035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82e1da-a1a3-4e3b-84bf-f411b5cea350 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:29.180310 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82e1da-a1a3-4e3b-84bf-f411b5cea350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:29.197978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-4d82e1da-a1a3-4e3b-84bf-f411b5cea350 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:29.199151 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-4d82e1da-a1a3-4e3b-84bf-f411b5cea350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:29.217075 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a67d30-9cf4-4d04-bacd-1a7a175e33a4 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:29.253004 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc50a8-1677-4779-80fc-fed3c5224c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:29.253533 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc50a8-1677-4779-80fc-fed3c5224c7e None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:29.253810 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc50a8-1677-4779-80fc-fed3c5224c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:29.254081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc50a8-1677-4779-80fc-fed3c5224c7e None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:29.254747 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbc50a8-1677-4779-80fc-fed3c5224c7e None admin] RBAC: Adding query filter params (name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:29.255047 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbc50a8-1677-4779-80fc-fed3c5224c7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:29.256685 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbc50a8-1677-4779-80fc-fed3c5224c7e None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:29.266653 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 421/849] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Aug 23 04:15:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:29.267268 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc50a8-1677-4779-80fc-fed3c5224c7e None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:29.278727 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 429/850] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:29 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:15:29.279466 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d82e1da-a1a3-4e3b-84bf-f411b5cea350 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:29.356075 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:29.359686 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:29.388017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d82e1da-a1a3-4e3b-84bf-f411b5cea350 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:29.415270 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc50a8-1677-4779-80fc-fed3c5224c7e None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:29.425693 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df439dfd-600c-46a1-aa10-f80f0bb6056a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:29.426036 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df439dfd-600c-46a1-aa10-f80f0bb6056a None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:29.426315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df439dfd-600c-46a1-aa10-f80f0bb6056a None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:29.426579 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df439dfd-600c-46a1-aa10-f80f0bb6056a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:29.439172 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df439dfd-600c-46a1-aa10-f80f0bb6056a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:29.441293 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df439dfd-600c-46a1-aa10-f80f0bb6056a None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:29.461777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-df439dfd-600c-46a1-aa10-f80f0bb6056a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:29.461777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:29.461777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:29.461777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:29.461777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:29.461777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:29.461777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:29.461777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:29.461777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:29.461777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:29.461777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:29.461777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:29.461777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:29.461777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:29.461777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:29.461777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:29.462705 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:29.462705 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:29.462705 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:29.462705 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:29.462705 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:29.462705 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:29.462705 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:29.462705 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:29.462705 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:29.462705 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:29.462705 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:29.462705 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:29.462705 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:29.462705 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:29.462705 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:29.462705 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:29.462705 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:29.463973 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:29.463973 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:29.463973 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:29.464158 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c640ea4-e464-4694-904e-4a49c78bf45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:29.464761 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c640ea4-e464-4694-904e-4a49c78bf45d None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:29.465042 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c640ea4-e464-4694-904e-4a49c78bf45d None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:29.465317 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c640ea4-e464-4694-904e-4a49c78bf45d None admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:29.466024 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c640ea4-e464-4694-904e-4a49c78bf45d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:29.466380 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c640ea4-e464-4694-904e-4a49c78bf45d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:29.467288 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 430/851] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:29.468595 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c640ea4-e464-4694-904e-4a49c78bf45d None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:29.472897 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df439dfd-600c-46a1-aa10-f80f0bb6056a None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:29.486235 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 422/852] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:15:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:29.492644 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c640ea4-e464-4694-904e-4a49c78bf45d None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:29.599402 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c640ea4-e464-4694-904e-4a49c78bf45d None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:29.603712 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df439dfd-600c-46a1-aa10-f80f0bb6056a None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:29.623163 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:29.632276 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:29.639649 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ddb46-92c2-4d26-b891-1e2661d828e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:29.640298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ddb46-92c2-4d26-b891-1e2661d828e4 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:29.640580 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ddb46-92c2-4d26-b891-1e2661d828e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:29.640845 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ddb46-92c2-4d26-b891-1e2661d828e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:29.641594 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8ddb46-92c2-4d26-b891-1e2661d828e4 None admin] RBAC: Adding query filter params (role.id=52b31061b72c45ccb76dd26bf31f8d3f, scope.project.id=9f4373dfb5f9425eb73669e24d18a08f, user.id=3f1418fd211a448da1a50175dce13b32) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:29.641885 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8ddb46-92c2-4d26-b891-1e2661d828e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:29.642202 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e026264-2edd-458c-a034-dc9434a27890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:29.642814 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e026264-2edd-458c-a034-dc9434a27890 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:29.643092 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e026264-2edd-458c-a034-dc9434a27890 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:29.643335 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8ddb46-92c2-4d26-b891-1e2661d828e4 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:29.643428 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e026264-2edd-458c-a034-dc9434a27890 None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:29.644016 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e026264-2edd-458c-a034-dc9434a27890 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:29.644321 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e026264-2edd-458c-a034-dc9434a27890 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:29.645674 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e026264-2edd-458c-a034-dc9434a27890 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:29.660301 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 423/853] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:15:29 2022] GET /identity/v3/role_assignments?user.id=3f1418fd211a448da1a50175dce13b32&scope.project.id=9f4373dfb5f9425eb73669e24d18a08f&role.id=52b31061b72c45ccb76dd26bf31f8d3f => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:29.660301 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ddb46-92c2-4d26-b891-1e2661d828e4 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:29.667365 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 431/854] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:29.673925 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e026264-2edd-458c-a034-dc9434a27890 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:29.774516 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ddb46-92c2-4d26-b891-1e2661d828e4 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:29.801003 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e026264-2edd-458c-a034-dc9434a27890 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:29.813501 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b1032-6ee8-4c3b-a1c4-9f4526ff1e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:29.814512 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b1032-6ee8-4c3b-a1c4-9f4526ff1e7d None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:29.814512 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b1032-6ee8-4c3b-a1c4-9f4526ff1e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:29.814695 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b1032-6ee8-4c3b-a1c4-9f4526ff1e7d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:29.815278 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b1032-6ee8-4c3b-a1c4-9f4526ff1e7d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:29.816714 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b1032-6ee8-4c3b-a1c4-9f4526ff1e7d None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:29.826813 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-9e0b1032-6ee8-4c3b-a1c4-9f4526ff1e7d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 04:15:29.826813 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:29.826813 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 04:15:29.826813 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 04:15:29.826813 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 04:15:29.826813 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 04:15:29.826813 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 04:15:29.826813 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:29.826813 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 04:15:29.826813 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:29.826813 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:15:29.826813 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:29.826813 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:29.826813 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:29.826813 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:29.826813 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:29.826813 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:29.827830 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:29.827830 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:29.827830 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:29.827830 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:29.827830 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:29.827830 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:29.827830 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 04:15:29.827830 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 04:15:29.827830 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 04:15:29.827830 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 04:15:29.827830 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:29.827830 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:29.827830 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:29.827830 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:29.827830 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:29.827830 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:29.828693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:29.828693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:29.828693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:29.828693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:29.828693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:29.828693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:29.828693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:29.828693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:29.828693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:29.828693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:29.828693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 04:15:29.828693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:29.828693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 04:15:29.828693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:29.828693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 424/855] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Aug 23 04:15:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:29.830006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74880432-5e2b-42f6-b982-ab0e59c7be39 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:29.830228 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74880432-5e2b-42f6-b982-ab0e59c7be39 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:29.830500 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74880432-5e2b-42f6-b982-ab0e59c7be39 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:29.834612 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 425/856] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:29.834612 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74880432-5e2b-42f6-b982-ab0e59c7be39 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:29.895503 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:29.956354 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74880432-5e2b-42f6-b982-ab0e59c7be39 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:29.962712 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76daa1-d5b4-4c73-9991-2b2944ebc891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:29.963722 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76daa1-d5b4-4c73-9991-2b2944ebc891 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:29.964025 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76daa1-d5b4-4c73-9991-2b2944ebc891 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:29.964300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76daa1-d5b4-4c73-9991-2b2944ebc891 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:29.972853 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce76daa1-d5b4-4c73-9991-2b2944ebc891 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:29.974376 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce76daa1-d5b4-4c73-9991-2b2944ebc891 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:29.985943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-ce76daa1-d5b4-4c73-9991-2b2944ebc891 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 04:15:29.985943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:29.985943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:29.985943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:29.985943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:29.985943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:29.985943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:29.985943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:29.985943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:29.985943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:29.985943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:29.985943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:29.985943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:15:29.985943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:15:29.985943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:15:29.985943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:15:29.986845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:29.986845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:29.986845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:15:29.986845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:29.986845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:15:29.986845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:29.986845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:29.986845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:29.986845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:29.986845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:29.986845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:29.986845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:29.986845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:29.986845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:29.986845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:29.986845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:29.986845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:29.986845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:29.987736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:29.987736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:29.987736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:15:29.987736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:15:29.987736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:15:29.987736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:15:29.987736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:15:29.987736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:15:29.987736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 04:15:29.987736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:29.987736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 432/857] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Aug 23 04:15:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:29.988779 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cb304-4cb2-48d4-95ac-65818024c69b None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:29.988779 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cb304-4cb2-48d4-95ac-65818024c69b None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:29.988901 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cb304-4cb2-48d4-95ac-65818024c69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:29.998661 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3af3c-2c2c-4a3e-8d20-445ffdd01265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:29.999275 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3af3c-2c2c-4a3e-8d20-445ffdd01265 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:29.999549 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3af3c-2c2c-4a3e-8d20-445ffdd01265 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:29.999815 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3af3c-2c2c-4a3e-8d20-445ffdd01265 None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:30.000477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e3af3c-2c2c-4a3e-8d20-445ffdd01265 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:30.000775 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e3af3c-2c2c-4a3e-8d20-445ffdd01265 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:30.002591 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e3af3c-2c2c-4a3e-8d20-445ffdd01265 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:30.006807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-ee7cb304-4cb2-48d4-95ac-65818024c69b None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:30.012255 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 426/858] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Aug 23 04:15:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:30.012397 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-ee7cb304-4cb2-48d4-95ac-65818024c69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:30.012491 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3af3c-2c2c-4a3e-8d20-445ffdd01265 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:30.055652 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:30.081416 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 433/859] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:29 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:15:30.082603 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cb304-4cb2-48d4-95ac-65818024c69b None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:30.150809 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3af3c-2c2c-4a3e-8d20-445ffdd01265 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:30.168674 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:30.192585 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab00f8-be26-4571-a613-8d25da987660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:30.193407 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab00f8-be26-4571-a613-8d25da987660 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:30.193871 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab00f8-be26-4571-a613-8d25da987660 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:30.194241 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab00f8-be26-4571-a613-8d25da987660 None admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:30.195082 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab00f8-be26-4571-a613-8d25da987660 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:30.195561 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab00f8-be26-4571-a613-8d25da987660 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:30.197141 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab00f8-be26-4571-a613-8d25da987660 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:30.201356 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cb304-4cb2-48d4-95ac-65818024c69b None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:30.216159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 427/860] 10.222.0.18 () {56 vars in 1149 bytes} [Tue Aug 23 04:15:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:30.221378 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab00f8-be26-4571-a613-8d25da987660 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:30.246576 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c6fa3-8fa5-468a-b463-de088ccfccd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:30.247221 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c6fa3-8fa5-468a-b463-de088ccfccd1 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:30.247500 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c6fa3-8fa5-468a-b463-de088ccfccd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:30.247773 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c6fa3-8fa5-468a-b463-de088ccfccd1 None admin] PATH_INFO: `/v3/services` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:30.248454 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137c6fa3-8fa5-468a-b463-de088ccfccd1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:30.248759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137c6fa3-8fa5-468a-b463-de088ccfccd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:30.250306 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137c6fa3-8fa5-468a-b463-de088ccfccd1 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:30.255750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 434/861] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Aug 23 04:15:30 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:30.346040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab00f8-be26-4571-a613-8d25da987660 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:30.365617 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c6fa3-8fa5-468a-b463-de088ccfccd1 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:30.390469 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ffe86-c22f-4a18-a5f2-786ec6617278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:30.392157 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ffe86-c22f-4a18-a5f2-786ec6617278 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:30.392574 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ffe86-c22f-4a18-a5f2-786ec6617278 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:30.392980 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ffe86-c22f-4a18-a5f2-786ec6617278 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:30.423430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:30.447596 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:30.476017 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93ffe86-c22f-4a18-a5f2-786ec6617278 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:30.478132 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93ffe86-c22f-4a18-a5f2-786ec6617278 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:30.493721 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c6fa3-8fa5-468a-b463-de088ccfccd1 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:30.542078 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec711f-927e-4c40-a1e6-de7a3c55adc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:30.542712 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec711f-927e-4c40-a1e6-de7a3c55adc1 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:30.542982 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec711f-927e-4c40-a1e6-de7a3c55adc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:30.543245 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec711f-927e-4c40-a1e6-de7a3c55adc1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:30.543818 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ec711f-927e-4c40-a1e6-de7a3c55adc1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:30.545200 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ec711f-927e-4c40-a1e6-de7a3c55adc1 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:30.555277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-a9ec711f-927e-4c40-a1e6-de7a3c55adc1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 04:15:30.555277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:30.555277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 04:15:30.555277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 04:15:30.555277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 04:15:30.555277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 04:15:30.555277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 04:15:30.555277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:30.555277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 04:15:30.555277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:30.555277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:15:30.555277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:30.555277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:30.555277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:30.555277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:30.555277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:30.556525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:30.556525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:30.556525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:30.556525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:30.556525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:30.556525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:30.556525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:30.556525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 04:15:30.556525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 04:15:30.556525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 04:15:30.556525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 04:15:30.556525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:30.556525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:30.556525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:30.556525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:30.556525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:30.556525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:30.557389 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:30.557389 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:30.557389 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:30.557389 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:30.557389 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:30.557389 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:30.557389 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:30.557389 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:30.557389 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:30.557389 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:30.557389 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 04:15:30.557389 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:30.557389 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 04:15:30.557389 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:30.557389 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 435/862] 10.222.0.18 () {56 vars in 1132 bytes} [Tue Aug 23 04:15:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:30.562367 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec711f-927e-4c40-a1e6-de7a3c55adc1 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:30.605657 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 428/863] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:15:30 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 04:15:30.607985 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ffe86-c22f-4a18-a5f2-786ec6617278 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:30.688813 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec711f-927e-4c40-a1e6-de7a3c55adc1 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:30.692320 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:30.693196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:30.730839 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ffe86-c22f-4a18-a5f2-786ec6617278 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:30.740603 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ccb4b6-43bb-48e6-a45b-cdd1af756eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:30.741607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccb4b6-43bb-48e6-a45b-cdd1af756eae None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:30.742496 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccb4b6-43bb-48e6-a45b-cdd1af756eae None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:30.742974 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccb4b6-43bb-48e6-a45b-cdd1af756eae None admin] PATH_INFO: `/v3/services` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:30.744156 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ccb4b6-43bb-48e6-a45b-cdd1af756eae None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:30.744947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ccb4b6-43bb-48e6-a45b-cdd1af756eae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:30.746726 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ccb4b6-43bb-48e6-a45b-cdd1af756eae None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:30.753729 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 436/864] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Aug 23 04:15:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:30.758993 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ccb4b6-43bb-48e6-a45b-cdd1af756eae None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:30.771943 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f73f4c-1609-4774-8948-f5c04e4125d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:30.772621 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f73f4c-1609-4774-8948-f5c04e4125d1 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:30.772961 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f73f4c-1609-4774-8948-f5c04e4125d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:30.773239 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f73f4c-1609-4774-8948-f5c04e4125d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:30.784632 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f73f4c-1609-4774-8948-f5c04e4125d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:30.785749 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f73f4c-1609-4774-8948-f5c04e4125d1 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:30.801182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-20f73f4c-1609-4774-8948-f5c04e4125d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:30.801182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:30.801182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:30.801182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:30.801182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:30.801182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:30.801182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:30.801182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:30.801182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:30.801182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:30.801182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:30.801182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:30.801182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:30.801182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:30.801182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:30.801182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:30.802426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:30.802426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:30.802426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:30.802426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:30.802426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:30.802426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:30.802426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:30.802426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:30.802426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:30.802426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:30.802426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:30.802426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:30.802426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:30.802426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:30.802426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:30.802426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:30.802426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:30.803541 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:30.803541 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:30.803541 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:30.807441 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 429/865] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:30.808466 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f73f4c-1609-4774-8948-f5c04e4125d1 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:30.874405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ccb4b6-43bb-48e6-a45b-cdd1af756eae None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:30.910796 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f73f4c-1609-4774-8948-f5c04e4125d1 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:30.911261 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11fac8-15df-4eaa-8068-97e23b2deea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:30.912169 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11fac8-15df-4eaa-8068-97e23b2deea2 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:30.912648 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11fac8-15df-4eaa-8068-97e23b2deea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:30.913196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11fac8-15df-4eaa-8068-97e23b2deea2 None admin] PATH_INFO: `/v3/services` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:30.914102 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11fac8-15df-4eaa-8068-97e23b2deea2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:30.914604 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11fac8-15df-4eaa-8068-97e23b2deea2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:30.916191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11fac8-15df-4eaa-8068-97e23b2deea2 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:30.921726 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 437/866] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Aug 23 04:15:30 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:30.942592 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:30.955697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c56e7b-3eb0-41af-a945-39c50a0391ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:30.956369 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c56e7b-3eb0-41af-a945-39c50a0391ff None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:30.956644 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c56e7b-3eb0-41af-a945-39c50a0391ff None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:30.956911 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c56e7b-3eb0-41af-a945-39c50a0391ff None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:30.957565 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c56e7b-3eb0-41af-a945-39c50a0391ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:30.957865 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c56e7b-3eb0-41af-a945-39c50a0391ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:30.959630 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c56e7b-3eb0-41af-a945-39c50a0391ff None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:30.973808 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 430/867] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:30.980052 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11fac8-15df-4eaa-8068-97e23b2deea2 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:31.017847 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:31.096178 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11fac8-15df-4eaa-8068-97e23b2deea2 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:31.133471 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69258d80-6889-41a8-a244-3d6881511d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:31.134469 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69258d80-6889-41a8-a244-3d6881511d6a None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:31.134926 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69258d80-6889-41a8-a244-3d6881511d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:31.135371 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69258d80-6889-41a8-a244-3d6881511d6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:31.142740 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69258d80-6889-41a8-a244-3d6881511d6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:31.144420 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69258d80-6889-41a8-a244-3d6881511d6a None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:31.156111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-69258d80-6889-41a8-a244-3d6881511d6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:31.156111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:31.156111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:31.156111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:31.156111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:31.156111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:31.156111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:31.156111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:31.156111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:31.156111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:31.156111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:31.156111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:31.156111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:15:31.156111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:15:31.156111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:15:31.157015 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:15:31.157015 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:31.157015 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:31.157015 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:31.157015 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:31.157015 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:31.157015 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:31.157015 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:31.157015 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:31.157015 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:31.157015 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:31.157015 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:31.157015 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:31.157015 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:31.157015 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:31.157015 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:31.157015 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:31.157904 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:15:31.157904 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:15:31.157904 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:15:31.157904 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:15:31.157904 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:15:31.157904 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:15:31.157904 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:31.157904 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:31.160027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 438/868] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:15:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:31.167949 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c56e7b-3eb0-41af-a945-39c50a0391ff None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:31.270222 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c56e7b-3eb0-41af-a945-39c50a0391ff None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:31.305237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a11c0-6620-44ee-8570-d3363feec26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:31.305915 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a11c0-6620-44ee-8570-d3363feec26c None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:31.306193 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a11c0-6620-44ee-8570-d3363feec26c None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:31.306463 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a11c0-6620-44ee-8570-d3363feec26c None admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:31.307207 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a11c0-6620-44ee-8570-d3363feec26c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:31.307516 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a11c0-6620-44ee-8570-d3363feec26c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:31.309007 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a11c0-6620-44ee-8570-d3363feec26c None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:31.321504 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 431/869] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:15:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:31.327446 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69258d80-6889-41a8-a244-3d6881511d6a None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:31.425154 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:31.434862 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69258d80-6889-41a8-a244-3d6881511d6a None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:31.469966 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5995336-8cd2-49dd-b1f0-773d4452b74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:31.470580 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5995336-8cd2-49dd-b1f0-773d4452b74d None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:31.470853 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5995336-8cd2-49dd-b1f0-773d4452b74d None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:31.471118 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5995336-8cd2-49dd-b1f0-773d4452b74d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:31.471843 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5995336-8cd2-49dd-b1f0-773d4452b74d None admin] RBAC: Adding query filter params (role.id=bdf71ef6ab10430e8c356cd4d952ff5c, scope.project.id=9f4373dfb5f9425eb73669e24d18a08f, user.id=d3538271ebfd41aabe9f3da85feb7bca) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:31.472169 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5995336-8cd2-49dd-b1f0-773d4452b74d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:31.473593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5995336-8cd2-49dd-b1f0-773d4452b74d None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:31.480359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 439/870] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:15:31 2022] GET /identity/v3/role_assignments?user.id=d3538271ebfd41aabe9f3da85feb7bca&scope.project.id=9f4373dfb5f9425eb73669e24d18a08f&role.id=bdf71ef6ab10430e8c356cd4d952ff5c => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:31.652302 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5e168-4463-4bce-8b9a-98d9abdf9955 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:31.652837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5e168-4463-4bce-8b9a-98d9abdf9955 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:31.652837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5e168-4463-4bce-8b9a-98d9abdf9955 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:31.654455 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 432/871] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:31.659802 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156b855-3a2d-450e-ac16-5d50e8b3c827 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:31.660128 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156b855-3a2d-450e-ac16-5d50e8b3c827 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:31.660396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156b855-3a2d-450e-ac16-5d50e8b3c827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:31.679230 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-0156b855-3a2d-450e-ac16-5d50e8b3c827 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:31.680447 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-0156b855-3a2d-450e-ac16-5d50e8b3c827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:31.751114 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 440/872] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:31 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:15:32.016516 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5e168-4463-4bce-8b9a-98d9abdf9955 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:32.062972 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:32.128896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5e168-4463-4bce-8b9a-98d9abdf9955 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:32.163200 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c23742-4386-4cc4-8527-be8ed5d110d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:32.163845 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c23742-4386-4cc4-8527-be8ed5d110d7 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:32.164137 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c23742-4386-4cc4-8527-be8ed5d110d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:32.164403 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c23742-4386-4cc4-8527-be8ed5d110d7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:32.164975 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c23742-4386-4cc4-8527-be8ed5d110d7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:32.166354 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c23742-4386-4cc4-8527-be8ed5d110d7 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:32.174858 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-58c23742-4386-4cc4-8527-be8ed5d110d7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 04:15:32.174858 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:32.174858 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 04:15:32.174858 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 04:15:32.174858 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 04:15:32.174858 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 04:15:32.174858 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 04:15:32.174858 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:32.174858 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 04:15:32.174858 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:32.174858 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:15:32.174858 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:32.174858 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:32.174858 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:32.174858 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:32.174858 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:32.174858 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:32.175816 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:32.175816 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:32.175816 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:32.175816 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:32.175816 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:32.175816 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:32.175816 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 04:15:32.175816 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 04:15:32.175816 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 04:15:32.175816 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 04:15:32.175816 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:32.175816 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:32.175816 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:32.175816 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:32.175816 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:32.175816 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:32.176891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:32.176891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:32.176891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:32.176891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:32.176891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:32.176891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:32.176891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:32.176891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:32.176891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:32.176891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:32.176891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 04:15:32.176891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:32.176891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 04:15:32.176891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:32.176891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 433/873] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Aug 23 04:15:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:32.182358 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0156b855-3a2d-450e-ac16-5d50e8b3c827 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:32.190520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e91dc-31c4-4ffa-965e-30393b56cfae None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:32.190808 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e91dc-31c4-4ffa-965e-30393b56cfae None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:32.191134 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e91dc-31c4-4ffa-965e-30393b56cfae None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:32.192491 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 434/874] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:32.197736 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64bd100-4d48-424c-b290-0753c8eef8dd None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:32.198017 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64bd100-4d48-424c-b290-0753c8eef8dd None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:32.198291 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64bd100-4d48-424c-b290-0753c8eef8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:32.220446 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-a64bd100-4d48-424c-b290-0753c8eef8dd None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:32.221653 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-a64bd100-4d48-424c-b290-0753c8eef8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:32.276546 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:32.297111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0156b855-3a2d-450e-ac16-5d50e8b3c827 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:32.307287 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 435/875] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:32 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 04:15:32.342261 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e034a7e-ee86-4379-a952-30af401aa568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:32.342912 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e034a7e-ee86-4379-a952-30af401aa568 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:32.343183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e034a7e-ee86-4379-a952-30af401aa568 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:32.343499 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e034a7e-ee86-4379-a952-30af401aa568 None admin] PATH_INFO: `/v3/services` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:32.344154 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e034a7e-ee86-4379-a952-30af401aa568 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:32.344452 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e034a7e-ee86-4379-a952-30af401aa568 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:32.345870 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e034a7e-ee86-4379-a952-30af401aa568 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:32.350970 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 441/876] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Aug 23 04:15:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:32.356463 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64bd100-4d48-424c-b290-0753c8eef8dd None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:32.396356 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:32.459637 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64bd100-4d48-424c-b290-0753c8eef8dd None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:32.494871 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5289e3e-71a1-4739-a709-fde474aaec21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:32.495483 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5289e3e-71a1-4739-a709-fde474aaec21 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:32.495760 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5289e3e-71a1-4739-a709-fde474aaec21 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:32.496065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5289e3e-71a1-4739-a709-fde474aaec21 None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:32.496693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5289e3e-71a1-4739-a709-fde474aaec21 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:32.496989 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5289e3e-71a1-4739-a709-fde474aaec21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:32.498411 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5289e3e-71a1-4739-a709-fde474aaec21 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:32.503222 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 436/877] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Aug 23 04:15:32 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:32.598863 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e034a7e-ee86-4379-a952-30af401aa568 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:32.713534 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e034a7e-ee86-4379-a952-30af401aa568 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:32.752551 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626b284-44e8-44af-aafa-abf0268bd60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:32.753552 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626b284-44e8-44af-aafa-abf0268bd60c None admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:32.754023 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626b284-44e8-44af-aafa-abf0268bd60c None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:32.754499 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626b284-44e8-44af-aafa-abf0268bd60c None admin] PATH_INFO: `/v3/services` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:32.755552 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c626b284-44e8-44af-aafa-abf0268bd60c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:32.757203 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c626b284-44e8-44af-aafa-abf0268bd60c None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:32.776329 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 442/878] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Aug 23 04:15:32 2022] POST /identity/v3/services => generated 251 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:15:32.952917 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5fa93-dfcc-443e-9c63-e8bf4d0bb14a None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:32.953933 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5fa93-dfcc-443e-9c63-e8bf4d0bb14a None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:32.954437 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5fa93-dfcc-443e-9c63-e8bf4d0bb14a None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:32.956242 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 437/879] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:32.961918 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831614d9-7f30-4be7-bbe3-c342b486e62e None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:32.962477 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831614d9-7f30-4be7-bbe3-c342b486e62e None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:32.962969 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831614d9-7f30-4be7-bbe3-c342b486e62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:32.983157 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-831614d9-7f30-4be7-bbe3-c342b486e62e None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:32.984434 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-831614d9-7f30-4be7-bbe3-c342b486e62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:32.987477 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:33.068058 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 443/880] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:32 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:15:33.328683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5fa93-dfcc-443e-9c63-e8bf4d0bb14a None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:33.429433 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:33.433383 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5fa93-dfcc-443e-9c63-e8bf4d0bb14a None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:33.471939 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b2f35-b7cb-4ad5-9aeb-f2aa974c43a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:33.471939 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b2f35-b7cb-4ad5-9aeb-f2aa974c43a6 None admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:33.471939 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b2f35-b7cb-4ad5-9aeb-f2aa974c43a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:33.471939 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b2f35-b7cb-4ad5-9aeb-f2aa974c43a6 None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:33.472334 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b2f35-b7cb-4ad5-9aeb-f2aa974c43a6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:33.472402 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b2f35-b7cb-4ad5-9aeb-f2aa974c43a6 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:33.493153 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 438/881] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Aug 23 04:15:33 2022] POST /identity/v3/services => generated 269 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:15:33.543048 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c23767-e578-4259-9b58-6adff0d790e8 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:33.543589 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c23767-e578-4259-9b58-6adff0d790e8 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:33.543715 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c23767-e578-4259-9b58-6adff0d790e8 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:33.545425 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 444/882] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:33.554141 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88512da6-210a-4b94-86da-9bd8fc0859f9 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:33.554461 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88512da6-210a-4b94-86da-9bd8fc0859f9 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:33.554689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88512da6-210a-4b94-86da-9bd8fc0859f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:33.575858 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-88512da6-210a-4b94-86da-9bd8fc0859f9 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:33.577186 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-88512da6-210a-4b94-86da-9bd8fc0859f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:33.659694 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 439/883] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:33 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:15:33.934057 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c23767-e578-4259-9b58-6adff0d790e8 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:34.041249 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c23767-e578-4259-9b58-6adff0d790e8 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:34.075774 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b0b877-fad0-44fc-bff1-31f434ab6b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:34.076434 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0b877-fad0-44fc-bff1-31f434ab6b1f None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:34.076756 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0b877-fad0-44fc-bff1-31f434ab6b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:34.077024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0b877-fad0-44fc-bff1-31f434ab6b1f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:34.087329 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b0b877-fad0-44fc-bff1-31f434ab6b1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:15:34.092297 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-84b0b877-fad0-44fc-bff1-31f434ab6b1f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 04:15:34.092297 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:34.092297 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:34.092297 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:34.092297 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:34.092297 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:34.092297 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:34.092297 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:34.092297 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:34.092297 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:34.092297 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:34.092297 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:34.092297 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:15:34.092297 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:15:34.092297 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:15:34.092297 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:15:34.093363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:15:34.093363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:15:34.093363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:15:34.093363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:15:34.093363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:34.093363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:34.093363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:34.093363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:34.093363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:34.093363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:34.093363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:34.093363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:34.093363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:34.093363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:34.093363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:34.093363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:34.094179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:34.094179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:34.094179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:34.094179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:34.094179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:15:34.094179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:15:34.094179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:15:34.094179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:15:34.094179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:15:34.094179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:15:34.094179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 04:15:34.094179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:34.096858 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 445/884] 10.222.0.18 () {56 vars in 1120 bytes} [Tue Aug 23 04:15:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:34.097160 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:34.103960 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88512da6-210a-4b94-86da-9bd8fc0859f9 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:34.188702 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:34.204917 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88512da6-210a-4b94-86da-9bd8fc0859f9 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:34.241776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99447989-4c1c-4cf8-8b88-fce22b160b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:34.242398 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99447989-4c1c-4cf8-8b88-fce22b160b60 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:34.242683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99447989-4c1c-4cf8-8b88-fce22b160b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:34.242953 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99447989-4c1c-4cf8-8b88-fce22b160b60 None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:34.243629 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99447989-4c1c-4cf8-8b88-fce22b160b60 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:34.243941 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99447989-4c1c-4cf8-8b88-fce22b160b60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:34.245332 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99447989-4c1c-4cf8-8b88-fce22b160b60 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:34.258690 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 440/885] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Aug 23 04:15:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:34.260143 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b0b877-fad0-44fc-bff1-31f434ab6b1f None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:34.358150 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b0b877-fad0-44fc-bff1-31f434ab6b1f None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:34.394741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e950419-749c-48d8-a4cc-77599c8d232e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:34.395383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e950419-749c-48d8-a4cc-77599c8d232e None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:34.395654 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e950419-749c-48d8-a4cc-77599c8d232e None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:34.395975 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e950419-749c-48d8-a4cc-77599c8d232e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:34.406674 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e950419-749c-48d8-a4cc-77599c8d232e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:34.408142 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e950419-749c-48d8-a4cc-77599c8d232e None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:34.422789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-8e950419-749c-48d8-a4cc-77599c8d232e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:34.422789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:34.422789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:34.422789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:34.422789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:34.422789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:34.422789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:34.422789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:34.422789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:34.422789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:34.422789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:34.422789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:34.422789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:34.422789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:34.422789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:34.422789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:34.423842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:34.423842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:34.423842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:34.423842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:34.423842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:34.423842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:34.423842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:34.423842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:34.423842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:34.423842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:34.423842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:34.423842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:34.423842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:34.423842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:34.423842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:34.423842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:34.423842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:34.424740 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:34.424740 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:34.424740 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:34.424740 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 446/886] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:34.430214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99447989-4c1c-4cf8-8b88-fce22b160b60 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:34.519623 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865418d-2279-421b-b9d4-58776748050f None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:34.519900 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865418d-2279-421b-b9d4-58776748050f None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:34.520211 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865418d-2279-421b-b9d4-58776748050f None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:34.522179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 447/887] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:15:34.528075 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39332d04-f21f-4348-8af1-a2b499d7edab None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:34.528292 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39332d04-f21f-4348-8af1-a2b499d7edab None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:34.528577 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39332d04-f21f-4348-8af1-a2b499d7edab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:34.540119 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99447989-4c1c-4cf8-8b88-fce22b160b60 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:34.546859 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-39332d04-f21f-4348-8af1-a2b499d7edab None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:34.548133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-39332d04-f21f-4348-8af1-a2b499d7edab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:34.577447 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3b354-24ca-4efb-9301-2d951ef71fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:34.578504 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3b354-24ca-4efb-9301-2d951ef71fec None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:34.579074 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3b354-24ca-4efb-9301-2d951ef71fec None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:34.579528 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3b354-24ca-4efb-9301-2d951ef71fec None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:34.580390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b3b354-24ca-4efb-9301-2d951ef71fec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:34.580873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b3b354-24ca-4efb-9301-2d951ef71fec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:34.582594 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b3b354-24ca-4efb-9301-2d951ef71fec None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:34.593934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:34.597706 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 441/888] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:34.604885 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3b354-24ca-4efb-9301-2d951ef71fec None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:34.616799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:34.625105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 448/889] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 04:15:34.713568 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3b354-24ca-4efb-9301-2d951ef71fec None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:34.751190 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5a6d11-96da-4245-979a-62cd5254d2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:34.751835 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a6d11-96da-4245-979a-62cd5254d2a7 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:34.752146 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a6d11-96da-4245-979a-62cd5254d2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:34.752422 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a6d11-96da-4245-979a-62cd5254d2a7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:34.760045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5a6d11-96da-4245-979a-62cd5254d2a7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:34.761507 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5a6d11-96da-4245-979a-62cd5254d2a7 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:34.773176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-dc5a6d11-96da-4245-979a-62cd5254d2a7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 04:15:34.773176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:34.773176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:34.773176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:34.773176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:34.773176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:34.773176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:34.773176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:34.773176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:34.773176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:34.773176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:34.773176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:34.773176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:15:34.773176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:15:34.773176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:15:34.773176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:15:34.774092 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:34.774092 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:34.774092 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:15:34.774092 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:34.774092 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:15:34.774092 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:34.774092 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:34.774092 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:34.774092 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:34.774092 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:34.774092 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:34.774092 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:34.774092 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:34.774092 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:34.774092 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:34.774092 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:34.774092 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:34.774092 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:34.775044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:34.775044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:34.775044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:15:34.775044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:15:34.775044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:15:34.775044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:15:34.775044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:15:34.775044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:15:34.775044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 04:15:34.775044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:34.779076 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 442/890] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Aug 23 04:15:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:15:34.780527 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39332d04-f21f-4348-8af1-a2b499d7edab None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:34.789232 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19f2d5-df2d-4e5c-ae2f-2a64a9b7754c None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:34.789519 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19f2d5-df2d-4e5c-ae2f-2a64a9b7754c None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:34.789796 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19f2d5-df2d-4e5c-ae2f-2a64a9b7754c None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:34.791368 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 443/891] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:15:34.796207 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed006043-a2e9-4062-be54-3f81f0adbd0d None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:34.796482 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed006043-a2e9-4062-be54-3f81f0adbd0d None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:34.796755 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed006043-a2e9-4062-be54-3f81f0adbd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:34.816074 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-ed006043-a2e9-4062-be54-3f81f0adbd0d None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:34.817276 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-ed006043-a2e9-4062-be54-3f81f0adbd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:34.885255 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:34.892172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 444/892] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:15:34.897498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39332d04-f21f-4348-8af1-a2b499d7edab None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:34.921814 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed006043-a2e9-4062-be54-3f81f0adbd0d None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:34.936232 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da191b-4a33-4e88-b115-f455e959e698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:34.937159 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da191b-4a33-4e88-b115-f455e959e698 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:34.937648 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da191b-4a33-4e88-b115-f455e959e698 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:34.938197 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da191b-4a33-4e88-b115-f455e959e698 None admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:34.939218 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da191b-4a33-4e88-b115-f455e959e698 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:34.939739 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da191b-4a33-4e88-b115-f455e959e698 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:34.941538 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da191b-4a33-4e88-b115-f455e959e698 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:34.960616 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 449/893] 10.222.0.18 () {56 vars in 1149 bytes} [Tue Aug 23 04:15:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:34.968186 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da191b-4a33-4e88-b115-f455e959e698 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:35.025612 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:35.045215 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed006043-a2e9-4062-be54-3f81f0adbd0d None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:35.102907 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475d49a-92a8-452e-876f-772e28caffb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:35.103535 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475d49a-92a8-452e-876f-772e28caffb0 None admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:35.103777 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475d49a-92a8-452e-876f-772e28caffb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:35.104110 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475d49a-92a8-452e-876f-772e28caffb0 None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:35.104742 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475d49a-92a8-452e-876f-772e28caffb0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:35.106158 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475d49a-92a8-452e-876f-772e28caffb0 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:35.111780 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da191b-4a33-4e88-b115-f455e959e698 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:35.132633 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 445/894] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Aug 23 04:15:34 2022] POST /identity/v3/services => generated 241 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:15:35.160898 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba4f86-9539-4c05-ae82-f0e953533f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:35.161806 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba4f86-9539-4c05-ae82-f0e953533f8a None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:35.162186 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba4f86-9539-4c05-ae82-f0e953533f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:35.162613 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba4f86-9539-4c05-ae82-f0e953533f8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:35.176668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475d49a-92a8-452e-876f-772e28caffb0 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:35.241982 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba4f86-9539-4c05-ae82-f0e953533f8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:35.243477 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba4f86-9539-4c05-ae82-f0e953533f8a None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:35.251175 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:35.287171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:35.316796 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475d49a-92a8-452e-876f-772e28caffb0 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:35.354300 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b9649-3f88-44a9-b5eb-00a2642a952d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:35.354977 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b9649-3f88-44a9-b5eb-00a2642a952d None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:35.355257 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b9649-3f88-44a9-b5eb-00a2642a952d None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:35.355529 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b9649-3f88-44a9-b5eb-00a2642a952d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:35.356866 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943b9649-3f88-44a9-b5eb-00a2642a952d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:35.358261 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943b9649-3f88-44a9-b5eb-00a2642a952d None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:35.367715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-943b9649-3f88-44a9-b5eb-00a2642a952d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 04:15:35.367715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:35.367715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 04:15:35.367715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 04:15:35.367715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 04:15:35.367715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 04:15:35.367715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 04:15:35.367715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:35.367715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 04:15:35.367715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:35.367715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:15:35.367715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:35.367715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:35.367715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:35.367715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:35.367715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:35.367715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:35.368897 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:35.368897 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:35.368897 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:35.368897 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:35.368897 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:35.368897 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:35.368897 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 04:15:35.368897 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 04:15:35.368897 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 04:15:35.368897 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 04:15:35.368897 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:35.368897 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:35.368897 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:35.368897 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:35.368897 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:35.368897 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:35.369986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:35.369986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:35.369986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:35.369986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:35.369986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:35.369986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:35.369986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:35.369986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:35.369986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:35.369986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:35.369986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 04:15:35.369986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:35.369986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 04:15:35.369986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:35.369986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 446/895] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Aug 23 04:15:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:15:35.374882 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b9649-3f88-44a9-b5eb-00a2642a952d None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:35.375690 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 450/896] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:15:34 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 04:15:35.380981 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba4f86-9539-4c05-ae82-f0e953533f8a None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:35.491517 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba4f86-9539-4c05-ae82-f0e953533f8a None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:35.495351 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b9649-3f88-44a9-b5eb-00a2642a952d None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:35.520212 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:35.525115 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:35.531957 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957920e-e05f-4e68-a8e0-eaa432fc077c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:35.532599 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957920e-e05f-4e68-a8e0-eaa432fc077c None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:35.532877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957920e-e05f-4e68-a8e0-eaa432fc077c None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:35.533148 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957920e-e05f-4e68-a8e0-eaa432fc077c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:35.534425 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccbe28d-9aa1-4426-b38d-7b9beaed971c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:35.535034 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccbe28d-9aa1-4426-b38d-7b9beaed971c None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:35.535309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccbe28d-9aa1-4426-b38d-7b9beaed971c None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:35.535578 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccbe28d-9aa1-4426-b38d-7b9beaed971c None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:35.536346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccbe28d-9aa1-4426-b38d-7b9beaed971c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:35.536651 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccbe28d-9aa1-4426-b38d-7b9beaed971c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:35.538044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccbe28d-9aa1-4426-b38d-7b9beaed971c None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:35.547384 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 447/897] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Aug 23 04:15:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:35.547384 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e35909-cef9-4089-8e30-54ceed4d1b89 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:35.547384 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e35909-cef9-4089-8e30-54ceed4d1b89 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:35.547384 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e35909-cef9-4089-8e30-54ceed4d1b89 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:35.547384 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 448/898] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:35.549127 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957920e-e05f-4e68-a8e0-eaa432fc077c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:35.549659 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e35909-cef9-4089-8e30-54ceed4d1b89 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:35.551022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957920e-e05f-4e68-a8e0-eaa432fc077c None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:35.567087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-b957920e-e05f-4e68-a8e0-eaa432fc077c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:35.567087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:35.567087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:35.567087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:35.567087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:35.567087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:35.567087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:35.567087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:35.567087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:35.567087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:35.567087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:35.567087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:35.567087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:35.567087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:35.567087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:35.567087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:35.568135 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:35.568135 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:35.568135 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:35.568135 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:35.568135 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:35.568135 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:35.568135 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:35.568135 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:35.568135 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:35.568135 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:35.568135 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:35.568135 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:35.568135 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:35.568135 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:35.568135 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:35.568135 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:35.568135 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:35.569043 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:35.569043 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:35.569043 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:35.572825 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 451/899] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:35.574002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40aa700-621c-491f-8895-a47df3c6a347 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:35.574287 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40aa700-621c-491f-8895-a47df3c6a347 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:35.574560 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40aa700-621c-491f-8895-a47df3c6a347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:35.594439 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-a40aa700-621c-491f-8895-a47df3c6a347 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:35.595741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a40aa700-621c-491f-8895-a47df3c6a347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:35.660157 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e35909-cef9-4089-8e30-54ceed4d1b89 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:35.679946 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 452/900] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:15:35.679946 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40aa700-621c-491f-8895-a47df3c6a347 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:35.696053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f0feb-1032-4793-8f73-dcf91c839928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:35.696853 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f0feb-1032-4793-8f73-dcf91c839928 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:35.697114 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f0feb-1032-4793-8f73-dcf91c839928 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:35.697433 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f0feb-1032-4793-8f73-dcf91c839928 None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:35.698137 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f0feb-1032-4793-8f73-dcf91c839928 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:35.698443 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f0feb-1032-4793-8f73-dcf91c839928 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:35.700016 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f0feb-1032-4793-8f73-dcf91c839928 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:35.710406 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 449/901] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Aug 23 04:15:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:35.711423 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f0feb-1032-4793-8f73-dcf91c839928 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:35.767686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:35.794243 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40aa700-621c-491f-8895-a47df3c6a347 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:35.813019 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:35.830322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f0feb-1032-4793-8f73-dcf91c839928 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:35.845972 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceee908-dc13-447f-a3b5-f014d5b8a894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:35.847515 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceee908-dc13-447f-a3b5-f014d5b8a894 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:35.848136 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceee908-dc13-447f-a3b5-f014d5b8a894 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:35.848587 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceee908-dc13-447f-a3b5-f014d5b8a894 None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:35.849799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceee908-dc13-447f-a3b5-f014d5b8a894 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:35.850655 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceee908-dc13-447f-a3b5-f014d5b8a894 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:35.852321 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceee908-dc13-447f-a3b5-f014d5b8a894 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:35.868686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220795c-ec09-4587-a554-9ed5a6aa5c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:35.869560 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220795c-ec09-4587-a554-9ed5a6aa5c95 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:35.869697 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 453/902] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:35.870145 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220795c-ec09-4587-a554-9ed5a6aa5c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:35.870568 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220795c-ec09-4587-a554-9ed5a6aa5c95 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:35.871333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220795c-ec09-4587-a554-9ed5a6aa5c95 None admin] RBAC: Adding query filter params (interface=public, service_id=0d1c075a57514afbbd7087324f423942, region_id=RegionOne) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:35.871696 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220795c-ec09-4587-a554-9ed5a6aa5c95 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:35.876091 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220795c-ec09-4587-a554-9ed5a6aa5c95 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:35.876226 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceee908-dc13-447f-a3b5-f014d5b8a894 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:35.880722 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 450/903] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Aug 23 04:15:35 2022] GET /identity/v3/endpoints?service_id=0d1c075a57514afbbd7087324f423942&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:35.887995 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220795c-ec09-4587-a554-9ed5a6aa5c95 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:35.989037 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220795c-ec09-4587-a554-9ed5a6aa5c95 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:35.998556 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceee908-dc13-447f-a3b5-f014d5b8a894 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:36.004209 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:36.027103 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f225d8-27ab-4ad6-afbd-7a3757ad62cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:36.027621 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f225d8-27ab-4ad6-afbd-7a3757ad62cf None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:36.027896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f225d8-27ab-4ad6-afbd-7a3757ad62cf None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:36.028353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f225d8-27ab-4ad6-afbd-7a3757ad62cf None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:36.028792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f225d8-27ab-4ad6-afbd-7a3757ad62cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:36.030157 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f225d8-27ab-4ad6-afbd-7a3757ad62cf None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:36.033318 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f08f7-a994-4753-810f-23d4a2e5a874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:36.033955 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f08f7-a994-4753-810f-23d4a2e5a874 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:36.034231 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f08f7-a994-4753-810f-23d4a2e5a874 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:36.034500 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f08f7-a994-4753-810f-23d4a2e5a874 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:36.044136 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 451/904] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Aug 23 04:15:35 2022] GET /identity/v3/services => generated 1497 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:15:36.044136 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f225d8-27ab-4ad6-afbd-7a3757ad62cf None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:36.049805 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684f08f7-a994-4753-810f-23d4a2e5a874 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:36.051271 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684f08f7-a994-4753-810f-23d4a2e5a874 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:36.064600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-684f08f7-a994-4753-810f-23d4a2e5a874 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:36.064600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:36.064600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:36.064600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:36.064600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:36.064600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:36.064600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:36.064600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:36.064600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:36.064600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:36.064600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:36.064600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:36.064600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:15:36.064600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:15:36.064600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:15:36.065670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:15:36.065670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:36.065670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:36.065670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:36.065670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:36.065670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:36.065670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:36.065670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:36.065670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:36.065670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:36.065670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:36.065670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:36.065670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:36.065670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:36.065670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:36.065670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:36.065670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:36.066525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:15:36.066525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:15:36.066525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:15:36.066525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:15:36.066525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:15:36.066525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:15:36.066525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:36.066525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:36.070979 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 454/905] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:15:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:36.072438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f08f7-a994-4753-810f-23d4a2e5a874 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:36.111891 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:36.184597 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f225d8-27ab-4ad6-afbd-7a3757ad62cf None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:36.186375 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f08f7-a994-4753-810f-23d4a2e5a874 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:36.224840 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d602271-a84b-4eb6-9f5a-ad098a819396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:36.225464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d602271-a84b-4eb6-9f5a-ad098a819396 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:36.225775 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d602271-a84b-4eb6-9f5a-ad098a819396 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:36.226128 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d602271-a84b-4eb6-9f5a-ad098a819396 None admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:36.226799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d602271-a84b-4eb6-9f5a-ad098a819396 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:36.227100 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d602271-a84b-4eb6-9f5a-ad098a819396 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:36.228556 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d602271-a84b-4eb6-9f5a-ad098a819396 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:36.228661 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25009a66-d8ba-4a21-b68c-5dc2e64ac9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:36.229212 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25009a66-d8ba-4a21-b68c-5dc2e64ac9d6 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:36.229481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25009a66-d8ba-4a21-b68c-5dc2e64ac9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:36.229758 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25009a66-d8ba-4a21-b68c-5dc2e64ac9d6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:36.230467 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25009a66-d8ba-4a21-b68c-5dc2e64ac9d6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:36.231999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25009a66-d8ba-4a21-b68c-5dc2e64ac9d6 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:36.241283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-25009a66-d8ba-4a21-b68c-5dc2e64ac9d6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 04:15:36.241283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:36.241283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 04:15:36.241283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 04:15:36.241283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 04:15:36.241283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 04:15:36.241283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 04:15:36.241283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:36.241283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 04:15:36.241283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:36.241283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:15:36.241283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:36.241283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:36.241283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:36.241283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:36.241283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:36.242223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:36.242223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:36.242223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:36.242223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:36.242223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:36.242223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:36.242223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:36.242223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 04:15:36.242223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 04:15:36.242223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 04:15:36.242223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 04:15:36.242223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:36.242223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:36.242223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:36.242223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:36.242223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:36.242223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:36.243089 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:36.243089 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:36.243089 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:36.243089 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:36.243089 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:36.243089 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:36.243089 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:36.243089 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:36.243089 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:36.243089 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:36.243089 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 04:15:36.243089 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:36.243089 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 04:15:36.243089 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:36.243089 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 452/906] 10.222.0.18 () {56 vars in 1132 bytes} [Tue Aug 23 04:15:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:36.244480 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 455/907] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:15:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:36.249643 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25009a66-d8ba-4a21-b68c-5dc2e64ac9d6 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:36.253831 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d602271-a84b-4eb6-9f5a-ad098a819396 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:36.300917 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:36.361271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25009a66-d8ba-4a21-b68c-5dc2e64ac9d6 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:36.367417 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d602271-a84b-4eb6-9f5a-ad098a819396 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:36.397314 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57009117-eec6-4855-a9b4-da33f6b2e31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:36.397923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57009117-eec6-4855-a9b4-da33f6b2e31e None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:36.398208 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57009117-eec6-4855-a9b4-da33f6b2e31e None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:36.398479 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57009117-eec6-4855-a9b4-da33f6b2e31e None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:36.399111 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57009117-eec6-4855-a9b4-da33f6b2e31e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:36.399418 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57009117-eec6-4855-a9b4-da33f6b2e31e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:36.400851 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57009117-eec6-4855-a9b4-da33f6b2e31e None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:36.405371 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca6f6b-2675-42fd-bcc4-3580c5ebe50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:36.410699 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca6f6b-2675-42fd-bcc4-3580c5ebe50a None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:36.410826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 453/908] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Aug 23 04:15:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:36.410904 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca6f6b-2675-42fd-bcc4-3580c5ebe50a None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:36.410904 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca6f6b-2675-42fd-bcc4-3580c5ebe50a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:36.410904 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ca6f6b-2675-42fd-bcc4-3580c5ebe50a None admin] RBAC: Adding query filter params (role.id=3631d69e831a474b807aeca180b214dc, scope.project.id=9f4373dfb5f9425eb73669e24d18a08f, user.id=d3538271ebfd41aabe9f3da85feb7bca) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:36.410904 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ca6f6b-2675-42fd-bcc4-3580c5ebe50a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:36.411762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57009117-eec6-4855-a9b4-da33f6b2e31e None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:36.412525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:36.414908 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ca6f6b-2675-42fd-bcc4-3580c5ebe50a None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:36.427961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 456/909] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:15:36 2022] GET /identity/v3/role_assignments?user.id=d3538271ebfd41aabe9f3da85feb7bca&scope.project.id=9f4373dfb5f9425eb73669e24d18a08f&role.id=3631d69e831a474b807aeca180b214dc => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:36.534674 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57009117-eec6-4855-a9b4-da33f6b2e31e None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:36.543001 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:36.573406 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272d582-e456-4103-8048-fcf6a1c8bd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:36.573948 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272d582-e456-4103-8048-fcf6a1c8bd12 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:36.574230 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272d582-e456-4103-8048-fcf6a1c8bd12 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:36.574573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272d582-e456-4103-8048-fcf6a1c8bd12 None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:36.575219 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272d582-e456-4103-8048-fcf6a1c8bd12 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:36.575576 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272d582-e456-4103-8048-fcf6a1c8bd12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:36.577022 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272d582-e456-4103-8048-fcf6a1c8bd12 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:36.583136 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 454/910] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Aug 23 04:15:36 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:36.589718 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca6f6b-2675-42fd-bcc4-3580c5ebe50a None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:36.697171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca6f6b-2675-42fd-bcc4-3580c5ebe50a None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:36.733260 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5c8d3-a7c8-4596-bf54-b18dd2bf4612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:36.733898 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5c8d3-a7c8-4596-bf54-b18dd2bf4612 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:36.734173 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5c8d3-a7c8-4596-bf54-b18dd2bf4612 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:36.734443 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5c8d3-a7c8-4596-bf54-b18dd2bf4612 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:36.735172 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e5c8d3-a7c8-4596-bf54-b18dd2bf4612 None admin] RBAC: Adding query filter params (interface=public, service_id=9999af83a7344321ac553f10db2193ee, region_id=RegionOne) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:36.735480 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e5c8d3-a7c8-4596-bf54-b18dd2bf4612 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:36.736913 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e5c8d3-a7c8-4596-bf54-b18dd2bf4612 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:36.745616 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 457/911] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Aug 23 04:15:36 2022] GET /identity/v3/endpoints?service_id=9999af83a7344321ac553f10db2193ee&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:36.751307 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272d582-e456-4103-8048-fcf6a1c8bd12 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:36.854484 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272d582-e456-4103-8048-fcf6a1c8bd12 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:36.888970 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d37719-c823-41af-b2b7-12fc2ee3a154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:36.889574 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d37719-c823-41af-b2b7-12fc2ee3a154 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:36.889848 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d37719-c823-41af-b2b7-12fc2ee3a154 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:36.890118 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d37719-c823-41af-b2b7-12fc2ee3a154 None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:36.890722 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d37719-c823-41af-b2b7-12fc2ee3a154 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:36.892125 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d37719-c823-41af-b2b7-12fc2ee3a154 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:36.898774 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 455/912] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Aug 23 04:15:36 2022] GET /identity/v3/services => generated 1497 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:15:37.145735 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23597441-a9a1-46ed-9edd-f39f632cc745 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:37.146183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23597441-a9a1-46ed-9edd-f39f632cc745 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:37.147010 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23597441-a9a1-46ed-9edd-f39f632cc745 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:37.148687 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 458/913] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:37.154033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f5ebc-40c1-4f6b-84e3-296f1ff604eb None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:37.154323 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f5ebc-40c1-4f6b-84e3-296f1ff604eb None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:37.154601 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f5ebc-40c1-4f6b-84e3-296f1ff604eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:37.173448 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-ee3f5ebc-40c1-4f6b-84e3-296f1ff604eb None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:37.174647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-ee3f5ebc-40c1-4f6b-84e3-296f1ff604eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:37.202358 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:37.250061 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 456/914] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:15:37.527473 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23597441-a9a1-46ed-9edd-f39f632cc745 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:37.554023 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:37.639132 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23597441-a9a1-46ed-9edd-f39f632cc745 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:37.673607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b484c-b37b-4c78-875c-fc1e2a6ed7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:37.674326 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b484c-b37b-4c78-875c-fc1e2a6ed7dd None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:37.674606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b484c-b37b-4c78-875c-fc1e2a6ed7dd None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:37.674878 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b484c-b37b-4c78-875c-fc1e2a6ed7dd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:37.675520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b484c-b37b-4c78-875c-fc1e2a6ed7dd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:37.677018 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b484c-b37b-4c78-875c-fc1e2a6ed7dd None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:37.686378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-0c6b484c-b37b-4c78-875c-fc1e2a6ed7dd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 04:15:37.686378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:37.686378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 04:15:37.686378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 04:15:37.686378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 04:15:37.686378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 04:15:37.686378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 04:15:37.686378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:37.686378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 04:15:37.686378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:37.686378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:15:37.686378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:37.686378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:37.686378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:37.686378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:37.686378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:37.686378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:37.687316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:37.687316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:37.687316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:37.687316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:37.687316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:37.687316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:37.687316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 04:15:37.687316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 04:15:37.687316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 04:15:37.687316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 04:15:37.687316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:37.687316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:37.687316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:37.687316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:37.687316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:37.687316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:37.689196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:37.689196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:37.689196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:37.689196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:37.689196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:37.689196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:37.689196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:37.689196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:37.689196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:37.689196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:37.689196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 04:15:37.689196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:37.689196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 04:15:37.689196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:37.689196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 459/915] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Aug 23 04:15:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:37.694473 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f5ebc-40c1-4f6b-84e3-296f1ff604eb None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:37.797789 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f5ebc-40c1-4f6b-84e3-296f1ff604eb None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:37.832126 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a6083-8f8c-417a-bfca-e262825a83a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:37.832744 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a6083-8f8c-417a-bfca-e262825a83a4 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:37.833021 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a6083-8f8c-417a-bfca-e262825a83a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:37.833923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a6083-8f8c-417a-bfca-e262825a83a4 None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:37.834581 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a6083-8f8c-417a-bfca-e262825a83a4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:37.834884 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a6083-8f8c-417a-bfca-e262825a83a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:37.836349 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a6083-8f8c-417a-bfca-e262825a83a4 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:37.842850 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 457/916] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Aug 23 04:15:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:37.848636 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b484c-b37b-4c78-875c-fc1e2a6ed7dd None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:37.938921 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:37.953510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b484c-b37b-4c78-875c-fc1e2a6ed7dd None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:37.988013 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00d6f4-68ed-42b2-a19a-70b570bf4503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:37.988628 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00d6f4-68ed-42b2-a19a-70b570bf4503 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:37.988901 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00d6f4-68ed-42b2-a19a-70b570bf4503 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:37.989169 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00d6f4-68ed-42b2-a19a-70b570bf4503 None admin] PATH_INFO: `/v3/services` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:37.989861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b00d6f4-68ed-42b2-a19a-70b570bf4503 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:37.990159 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b00d6f4-68ed-42b2-a19a-70b570bf4503 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:37.991709 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b00d6f4-68ed-42b2-a19a-70b570bf4503 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:37.996893 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 460/917] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Aug 23 04:15:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:38.002842 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a6083-8f8c-417a-bfca-e262825a83a4 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:38.035064 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:38.079029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3ec5c-adae-4d48-be07-638d04f7c674 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:38.079638 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3ec5c-adae-4d48-be07-638d04f7c674 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:38.080151 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3ec5c-adae-4d48-be07-638d04f7c674 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:38.082949 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 461/918] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:15:38.087491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff882471-5bd6-46b4-8fef-9ec459704c07 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:38.087772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff882471-5bd6-46b4-8fef-9ec459704c07 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:38.088429 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff882471-5bd6-46b4-8fef-9ec459704c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:38.109906 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a6083-8f8c-417a-bfca-e262825a83a4 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:38.112130 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-ff882471-5bd6-46b4-8fef-9ec459704c07 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:38.113538 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-ff882471-5bd6-46b4-8fef-9ec459704c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:38.150802 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7bbef-49bb-44c2-b9ce-7a9341c866fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:38.151414 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7bbef-49bb-44c2-b9ce-7a9341c866fc None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:38.152296 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7bbef-49bb-44c2-b9ce-7a9341c866fc None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:38.152574 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7bbef-49bb-44c2-b9ce-7a9341c866fc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:38.153299 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7bbef-49bb-44c2-b9ce-7a9341c866fc None admin] RBAC: Adding query filter params (interface=public, service_id=3bc2958100e04ff3b7239b5ea9bdf191, region_id=RegionOne) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:38.153605 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7bbef-49bb-44c2-b9ce-7a9341c866fc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:38.155021 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7bbef-49bb-44c2-b9ce-7a9341c866fc None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:38.165317 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 458/919] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Aug 23 04:15:38 2022] GET /identity/v3/endpoints?service_id=3bc2958100e04ff3b7239b5ea9bdf191&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:38.167291 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7bbef-49bb-44c2-b9ce-7a9341c866fc None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:38.198650 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 462/920] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 04:15:38.267792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:38.277507 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7bbef-49bb-44c2-b9ce-7a9341c866fc None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:38.311487 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa897af5-baf1-4a9a-8270-34507bd06a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:38.312493 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa897af5-baf1-4a9a-8270-34507bd06a68 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:38.312991 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa897af5-baf1-4a9a-8270-34507bd06a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:38.313463 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa897af5-baf1-4a9a-8270-34507bd06a68 None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:38.314281 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa897af5-baf1-4a9a-8270-34507bd06a68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:38.315850 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa897af5-baf1-4a9a-8270-34507bd06a68 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:38.324012 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 459/921] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Aug 23 04:15:38 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:15:38.484076 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff882471-5bd6-46b4-8fef-9ec459704c07 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:38.506358 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da957c4-0581-4dd6-8047-e46a8d06dc23 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:38.506917 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da957c4-0581-4dd6-8047-e46a8d06dc23 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:38.507189 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da957c4-0581-4dd6-8047-e46a8d06dc23 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:38.509029 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 460/922] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:15:38.514736 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b9595-4866-46db-917d-4e453516422b None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:38.515255 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b9595-4866-46db-917d-4e453516422b None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:38.515792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b9595-4866-46db-917d-4e453516422b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:38.538711 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-840b9595-4866-46db-917d-4e453516422b None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:38.539908 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-840b9595-4866-46db-917d-4e453516422b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:38.571323 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:38.593112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff882471-5bd6-46b4-8fef-9ec459704c07 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:38.627780 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 461/923] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 04:15:38.641739 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c821f-33dd-4169-b376-41401af44a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:38.642965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c821f-33dd-4169-b376-41401af44a79 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:38.643405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c821f-33dd-4169-b376-41401af44a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:38.643827 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c821f-33dd-4169-b376-41401af44a79 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:38.644593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c821f-33dd-4169-b376-41401af44a79 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:38.646449 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c821f-33dd-4169-b376-41401af44a79 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:38.655807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-7e5c821f-33dd-4169-b376-41401af44a79 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 04:15:38.655807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:38.655807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 04:15:38.655807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 04:15:38.655807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 04:15:38.655807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 04:15:38.655807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 04:15:38.655807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:38.655807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 04:15:38.655807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:38.655807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:15:38.655807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:38.655807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:38.655807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:38.655807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:38.655807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:38.655807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:38.657120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:38.657120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:38.657120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:38.657120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:38.657120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:38.657120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:38.657120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 04:15:38.657120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 04:15:38.657120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 04:15:38.657120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 04:15:38.657120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:38.657120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:38.657120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:38.657120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:38.657120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:38.657120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:38.658246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:38.658246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:38.658246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:38.658246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:38.658246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:38.658246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:38.658246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:38.658246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:38.658246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:38.658246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:38.658246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 04:15:38.658246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:38.658246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 04:15:38.658246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:38.665301 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 463/924] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Aug 23 04:15:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:38.666845 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b9595-4866-46db-917d-4e453516422b None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:38.757772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:38.772954 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b9595-4866-46db-917d-4e453516422b None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:38.809371 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1180e585-4744-48cf-939b-887fd0ee736e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:38.809993 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180e585-4744-48cf-939b-887fd0ee736e None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:38.810273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180e585-4744-48cf-939b-887fd0ee736e None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:38.810544 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180e585-4744-48cf-939b-887fd0ee736e None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:38.811178 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1180e585-4744-48cf-939b-887fd0ee736e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:38.811479 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1180e585-4744-48cf-939b-887fd0ee736e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:38.812881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1180e585-4744-48cf-939b-887fd0ee736e None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:38.819751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 462/925] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Aug 23 04:15:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:38.825549 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c821f-33dd-4169-b376-41401af44a79 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:38.899203 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a452b49-6889-4e56-a6e2-cb018c5029e7 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:38.899595 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a452b49-6889-4e56-a6e2-cb018c5029e7 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:38.899965 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a452b49-6889-4e56-a6e2-cb018c5029e7 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:38.901569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 463/926] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:38.906478 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8b5a6-78f7-46f3-a5ab-788a4f48dd2b None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:38.906758 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8b5a6-78f7-46f3-a5ab-788a4f48dd2b None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:38.907030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8b5a6-78f7-46f3-a5ab-788a4f48dd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:38.926413 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-11d8b5a6-78f7-46f3-a5ab-788a4f48dd2b None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:38.928314 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-11d8b5a6-78f7-46f3-a5ab-788a4f48dd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:38.940652 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c821f-33dd-4169-b376-41401af44a79 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:38.977868 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc652b5d-902e-4bcc-a6bd-d4dfbb03c04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:38.978500 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc652b5d-902e-4bcc-a6bd-d4dfbb03c04d None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:38.978777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc652b5d-902e-4bcc-a6bd-d4dfbb03c04d None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:38.979048 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc652b5d-902e-4bcc-a6bd-d4dfbb03c04d None admin] PATH_INFO: `/v3/services` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:38.979689 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc652b5d-902e-4bcc-a6bd-d4dfbb03c04d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:38.980039 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc652b5d-902e-4bcc-a6bd-d4dfbb03c04d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:38.981394 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc652b5d-902e-4bcc-a6bd-d4dfbb03c04d None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:38.986345 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 464/927] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Aug 23 04:15:38 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:38.986345 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc652b5d-902e-4bcc-a6bd-d4dfbb03c04d None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:39.001878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 464/928] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 04:15:39.002701 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8b5a6-78f7-46f3-a5ab-788a4f48dd2b None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:39.004767 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:39.101587 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc652b5d-902e-4bcc-a6bd-d4dfbb03c04d None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:39.121974 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:39.139411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f4cac-e32e-4e29-b02c-1a0cede2e7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:39.140089 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f4cac-e32e-4e29-b02c-1a0cede2e7d2 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:39.140375 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f4cac-e32e-4e29-b02c-1a0cede2e7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:39.140647 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f4cac-e32e-4e29-b02c-1a0cede2e7d2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:39.148627 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8b5a6-78f7-46f3-a5ab-788a4f48dd2b None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:39.153098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f4cac-e32e-4e29-b02c-1a0cede2e7d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:15:39.158354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-be9f4cac-e32e-4e29-b02c-1a0cede2e7d2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 04:15:39.158354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:39.158354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:39.158354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:39.158354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:39.158354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:39.158354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:39.158354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:39.158354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:39.158354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:39.158354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:39.158354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:39.158354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:15:39.158354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:15:39.158354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:15:39.158354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:15:39.159288 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:15:39.159288 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:15:39.159288 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:15:39.159288 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:15:39.159288 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:39.159288 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:39.159288 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:39.159288 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:39.159288 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:39.159288 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:39.159288 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:39.159288 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:39.159288 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:39.159288 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:39.159288 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:39.159288 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:39.160324 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:39.160324 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:39.160324 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:39.160324 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:39.160324 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:15:39.160324 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:15:39.160324 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:15:39.160324 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:15:39.160324 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:15:39.160324 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:15:39.160324 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 04:15:39.160324 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:39.160324 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 465/929] 10.222.0.18 () {56 vars in 1120 bytes} [Tue Aug 23 04:15:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:39.166549 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f4cac-e32e-4e29-b02c-1a0cede2e7d2 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:39.189439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59463f7f-75b2-431f-ade0-7700da6a017f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:39.190427 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59463f7f-75b2-431f-ade0-7700da6a017f None admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:39.190925 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59463f7f-75b2-431f-ade0-7700da6a017f None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:39.191404 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59463f7f-75b2-431f-ade0-7700da6a017f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:39.192299 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59463f7f-75b2-431f-ade0-7700da6a017f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:39.194106 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59463f7f-75b2-431f-ade0-7700da6a017f None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:39.228007 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 465/930] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:15:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:15:39.253348 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:39.283072 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59463f7f-75b2-431f-ade0-7700da6a017f None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:39.295895 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f4cac-e32e-4e29-b02c-1a0cede2e7d2 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:39.347022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f983ec-6cce-4ba3-93c1-6a87135187ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:39.347736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f983ec-6cce-4ba3-93c1-6a87135187ef None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:39.348061 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f983ec-6cce-4ba3-93c1-6a87135187ef None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:39.348396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f983ec-6cce-4ba3-93c1-6a87135187ef None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:39.349223 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f983ec-6cce-4ba3-93c1-6a87135187ef None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:39.349519 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f983ec-6cce-4ba3-93c1-6a87135187ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:39.351032 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f983ec-6cce-4ba3-93c1-6a87135187ef None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:39.361415 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 466/931] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Aug 23 04:15:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:39.367240 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f983ec-6cce-4ba3-93c1-6a87135187ef None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:39.403891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59463f7f-75b2-431f-ade0-7700da6a017f None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:39.444176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9339b0-34c4-48ac-8bf9-f31ecf8ff3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:39.444823 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9339b0-34c4-48ac-8bf9-f31ecf8ff3c2 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:39.445101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9339b0-34c4-48ac-8bf9-f31ecf8ff3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:39.445372 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9339b0-34c4-48ac-8bf9-f31ecf8ff3c2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:39.445960 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9339b0-34c4-48ac-8bf9-f31ecf8ff3c2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:39.447346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9339b0-34c4-48ac-8bf9-f31ecf8ff3c2 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:39.450113 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:39.460108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-ab9339b0-34c4-48ac-8bf9-f31ecf8ff3c2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 04:15:39.460108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:39.460108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 04:15:39.460108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 04:15:39.460108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 04:15:39.460108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 04:15:39.460108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 04:15:39.460108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:39.460108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 04:15:39.460108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:39.460108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:15:39.460108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:39.460108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:39.460108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:39.460108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:39.460108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:39.461135 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:39.461135 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:39.461135 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:39.461135 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:39.461135 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:39.461135 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:39.461135 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:39.461135 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 04:15:39.461135 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 04:15:39.461135 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 04:15:39.461135 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 04:15:39.461135 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:39.461135 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:39.461135 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:39.461135 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:39.461135 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:39.461135 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:39.461999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:39.461999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:39.461999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:39.461999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:39.461999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:39.461999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:39.461999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:39.461999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:39.461999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:39.461999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:39.461999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 04:15:39.461999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:39.461999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 04:15:39.461999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:39.461999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 466/932] 10.222.0.18 () {56 vars in 1132 bytes} [Tue Aug 23 04:15:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:15:39.469579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9339b0-34c4-48ac-8bf9-f31ecf8ff3c2 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:39.502458 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f983ec-6cce-4ba3-93c1-6a87135187ef None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:39.540306 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:39.553665 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb5542-3a0d-4ba8-99e9-6e570db49189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:39.554449 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb5542-3a0d-4ba8-99e9-6e570db49189 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:39.554824 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb5542-3a0d-4ba8-99e9-6e570db49189 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:39.555188 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb5542-3a0d-4ba8-99e9-6e570db49189 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:39.570427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb5542-3a0d-4ba8-99e9-6e570db49189 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:39.572548 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb5542-3a0d-4ba8-99e9-6e570db49189 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:39.586062 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9339b0-34c4-48ac-8bf9-f31ecf8ff3c2 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:39.593139 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-c6bb5542-3a0d-4ba8-99e9-6e570db49189 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:39.593139 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:39.593139 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:39.593139 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:39.593139 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:39.593139 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:39.593139 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:39.593139 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:39.593139 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:39.593139 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:39.593139 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:39.593139 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:39.593139 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:39.593139 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:39.593139 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:39.593139 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:39.594069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:39.594069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:39.594069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:39.594069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:39.594069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:39.594069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:39.594069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:39.594069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:39.594069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:39.594069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:39.594069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:39.594069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:39.594069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:39.594069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:39.594069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:39.594069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:39.594069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:39.594965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:39.594965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:39.594965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:39.597353 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 467/933] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:15:39.602609 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb5542-3a0d-4ba8-99e9-6e570db49189 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:39.631338 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b3ff3-641d-4e25-8bcd-a4de93726007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:39.632407 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b3ff3-641d-4e25-8bcd-a4de93726007 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:39.632898 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b3ff3-641d-4e25-8bcd-a4de93726007 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:39.633443 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b3ff3-641d-4e25-8bcd-a4de93726007 None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:39.634432 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b3ff3-641d-4e25-8bcd-a4de93726007 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:39.634977 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b3ff3-641d-4e25-8bcd-a4de93726007 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:39.637046 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b3ff3-641d-4e25-8bcd-a4de93726007 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:39.642862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 467/934] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Aug 23 04:15:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:39.649897 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b3ff3-641d-4e25-8bcd-a4de93726007 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:39.712123 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb5542-3a0d-4ba8-99e9-6e570db49189 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:39.729661 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:39.752398 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ddf922-5522-435a-9772-aadb95962288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:39.753381 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ddf922-5522-435a-9772-aadb95962288 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:39.753808 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ddf922-5522-435a-9772-aadb95962288 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:39.754221 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ddf922-5522-435a-9772-aadb95962288 None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:39.755278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ddf922-5522-435a-9772-aadb95962288 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:39.756227 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ddf922-5522-435a-9772-aadb95962288 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:39.758454 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ddf922-5522-435a-9772-aadb95962288 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:39.776712 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 468/935] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:39.777932 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ddf922-5522-435a-9772-aadb95962288 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:39.778354 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b3ff3-641d-4e25-8bcd-a4de93726007 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:39.817145 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451c4b6-ff12-4a81-80b3-20c5ec5af0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:39.818233 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451c4b6-ff12-4a81-80b3-20c5ec5af0d0 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:39.818721 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451c4b6-ff12-4a81-80b3-20c5ec5af0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:39.819191 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451c4b6-ff12-4a81-80b3-20c5ec5af0d0 None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:39.820073 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451c4b6-ff12-4a81-80b3-20c5ec5af0d0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:39.820566 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451c4b6-ff12-4a81-80b3-20c5ec5af0d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:39.822171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451c4b6-ff12-4a81-80b3-20c5ec5af0d0 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:39.827791 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 468/936] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Aug 23 04:15:39 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:39.828409 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:39.834256 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451c4b6-ff12-4a81-80b3-20c5ec5af0d0 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:39.900702 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ddf922-5522-435a-9772-aadb95962288 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:39.938412 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70015d-e8de-4fd8-88b8-36c0a001ebd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:39.939525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70015d-e8de-4fd8-88b8-36c0a001ebd0 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:39.940019 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70015d-e8de-4fd8-88b8-36c0a001ebd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:39.940498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70015d-e8de-4fd8-88b8-36c0a001ebd0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:39.948515 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70015d-e8de-4fd8-88b8-36c0a001ebd0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:39.950234 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70015d-e8de-4fd8-88b8-36c0a001ebd0 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:39.964362 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451c4b6-ff12-4a81-80b3-20c5ec5af0d0 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:39.965420 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-8d70015d-e8de-4fd8-88b8-36c0a001ebd0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 04:15:39.965420 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:39.965420 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:39.965420 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:39.965420 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:39.965420 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:39.965420 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:39.965420 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:39.965420 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:39.965420 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:39.965420 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:39.965420 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:39.965420 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:15:39.965420 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:15:39.965420 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:15:39.965420 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:15:39.966300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:39.966300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:39.966300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:15:39.966300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:39.966300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:15:39.966300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:39.966300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:39.966300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:39.966300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:39.966300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:39.966300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:39.966300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:39.966300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:39.966300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:39.966300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:39.966300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:39.966300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:39.966300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:39.967215 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:39.967215 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:39.967215 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:15:39.967215 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:15:39.967215 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:15:39.967215 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:15:39.967215 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:15:39.967215 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:15:39.967215 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 04:15:39.967215 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:39.974413 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 469/937] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Aug 23 04:15:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:39.975703 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70015d-e8de-4fd8-88b8-36c0a001ebd0 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:39.995280 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:40.004233 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb62d9f-5ea8-4fbf-8f04-677f354542df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:40.004856 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb62d9f-5ea8-4fbf-8f04-677f354542df None admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:40.005161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb62d9f-5ea8-4fbf-8f04-677f354542df None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:40.005435 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb62d9f-5ea8-4fbf-8f04-677f354542df None admin] PATH_INFO: `/v3/endpoints` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:40.006129 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb62d9f-5ea8-4fbf-8f04-677f354542df None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:40.007534 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb62d9f-5ea8-4fbf-8f04-677f354542df None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:40.036082 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 469/938] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:15:39 2022] POST /identity/v3/endpoints => generated 347 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:15:40.089358 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:40.094605 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70015d-e8de-4fd8-88b8-36c0a001ebd0 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:40.138344 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b86188-d809-482d-be97-fa22abd69081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:40.139414 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b86188-d809-482d-be97-fa22abd69081 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:40.139894 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b86188-d809-482d-be97-fa22abd69081 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:40.140407 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b86188-d809-482d-be97-fa22abd69081 None admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:40.141362 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b86188-d809-482d-be97-fa22abd69081 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:40.141875 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b86188-d809-482d-be97-fa22abd69081 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:40.143548 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b86188-d809-482d-be97-fa22abd69081 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:40.169871 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 470/939] 10.222.0.18 () {56 vars in 1149 bytes} [Tue Aug 23 04:15:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:40.175610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb62d9f-5ea8-4fbf-8f04-677f354542df None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:40.290676 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb62d9f-5ea8-4fbf-8f04-677f354542df None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:40.295535 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ac8fd-868e-47e5-a462-9af5b21da962 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:40.295820 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ac8fd-868e-47e5-a462-9af5b21da962 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:40.296115 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ac8fd-868e-47e5-a462-9af5b21da962 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:40.297580 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 471/940] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:40.303314 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02aa275-e323-47a0-9aa2-6bc831f9f621 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:40.303875 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02aa275-e323-47a0-9aa2-6bc831f9f621 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:40.304370 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02aa275-e323-47a0-9aa2-6bc831f9f621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:40.323694 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-c02aa275-e323-47a0-9aa2-6bc831f9f621 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:40.325245 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-c02aa275-e323-47a0-9aa2-6bc831f9f621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:40.327152 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbd5de-5c13-460a-844a-d398c653e4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:40.327947 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbd5de-5c13-460a-844a-d398c653e4f7 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:40.328226 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbd5de-5c13-460a-844a-d398c653e4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:40.328500 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbd5de-5c13-460a-844a-d398c653e4f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:40.341192 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bbd5de-5c13-460a-844a-d398c653e4f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:40.342860 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bbd5de-5c13-460a-844a-d398c653e4f7 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:40.359030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-94bbd5de-5c13-460a-844a-d398c653e4f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:40.359030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:40.359030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:40.359030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:40.359030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:40.359030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:40.359030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:40.359030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:40.359030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:40.359030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:40.359030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:40.359030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:40.359030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:40.359030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:40.359030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:40.359030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:40.360087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:40.360087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:40.360087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:40.360087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:40.360087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:40.360087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:40.360087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:40.360087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:40.360087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:40.360087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:40.360087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:40.360087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:40.360087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:40.360087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:40.360087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:40.360087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:40.360087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:40.360953 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:40.360953 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:40.360953 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:40.360953 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 470/941] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:40.369588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbd5de-5c13-460a-844a-d398c653e4f7 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:40.403723 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 472/942] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 04:15:40.411808 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:40.475342 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbd5de-5c13-460a-844a-d398c653e4f7 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:40.509226 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d3d34-412a-4407-9769-a7847018da0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:40.509835 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d3d34-412a-4407-9769-a7847018da0a None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:40.510112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d3d34-412a-4407-9769-a7847018da0a None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:40.510376 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d3d34-412a-4407-9769-a7847018da0a None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:40.511018 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d3d34-412a-4407-9769-a7847018da0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:40.511316 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d3d34-412a-4407-9769-a7847018da0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:40.512712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d3d34-412a-4407-9769-a7847018da0a None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:40.524578 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 471/943] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:40.531069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02aa275-e323-47a0-9aa2-6bc831f9f621 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:40.632327 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:40.640702 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02aa275-e323-47a0-9aa2-6bc831f9f621 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:40.677769 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd4d4d5-53cc-4631-911c-3664fa851a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:40.678802 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4d4d5-53cc-4631-911c-3664fa851a0a None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:40.679321 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4d4d5-53cc-4631-911c-3664fa851a0a None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:40.679805 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4d4d5-53cc-4631-911c-3664fa851a0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:40.686891 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd4d4d5-53cc-4631-911c-3664fa851a0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:40.688662 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd4d4d5-53cc-4631-911c-3664fa851a0a None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:40.693625 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d3d34-412a-4407-9769-a7847018da0a None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:40.701008 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-8bd4d4d5-53cc-4631-911c-3664fa851a0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:40.701008 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:40.701008 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:40.701008 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:40.701008 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:40.701008 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:40.701008 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:40.701008 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:40.701008 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:40.701008 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:40.701008 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:40.701008 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:40.701008 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:15:40.701008 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:15:40.701008 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:15:40.701889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:15:40.701889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:40.701889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:40.701889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:40.701889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:40.701889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:40.701889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:40.701889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:40.701889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:40.701889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:40.701889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:40.701889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:40.701889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:40.701889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:40.701889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:40.701889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:40.701889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:40.702772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:15:40.702772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:15:40.702772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:15:40.702772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:15:40.702772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:15:40.702772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:15:40.702772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:40.702772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:40.705281 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 473/944] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:15:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:15:40.711217 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd4d4d5-53cc-4631-911c-3664fa851a0a None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:40.805457 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:40.812576 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d3d34-412a-4407-9769-a7847018da0a None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:40.821622 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd4d4d5-53cc-4631-911c-3664fa851a0a None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:40.850133 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea5b9e-e821-424d-8c05-0377d6ab7791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:40.851464 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea5b9e-e821-424d-8c05-0377d6ab7791 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:40.852017 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea5b9e-e821-424d-8c05-0377d6ab7791 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:40.852480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea5b9e-e821-424d-8c05-0377d6ab7791 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:40.853574 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea5b9e-e821-424d-8c05-0377d6ab7791 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:40.855619 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea5b9e-e821-424d-8c05-0377d6ab7791 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:40.858776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0b390-1c2b-4c84-8849-93e45978d930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:40.859328 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0b390-1c2b-4c84-8849-93e45978d930 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:40.859797 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0b390-1c2b-4c84-8849-93e45978d930 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:40.860026 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0b390-1c2b-4c84-8849-93e45978d930 None admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:40.860750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0b390-1c2b-4c84-8849-93e45978d930 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:40.861052 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0b390-1c2b-4c84-8849-93e45978d930 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:40.862478 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0b390-1c2b-4c84-8849-93e45978d930 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:40.867873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-27ea5b9e-e821-424d-8c05-0377d6ab7791 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 04:15:40.867873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:40.867873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 04:15:40.867873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 04:15:40.867873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 04:15:40.867873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 04:15:40.867873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 04:15:40.867873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:40.867873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 04:15:40.867873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:40.867873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:15:40.867873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:40.867873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:40.867873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:40.867873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:40.867873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:40.867873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:40.868875 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:40.868875 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:40.868875 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:40.868875 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:40.868875 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:40.868875 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:40.868875 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 04:15:40.868875 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 04:15:40.868875 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 04:15:40.868875 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 04:15:40.868875 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:40.868875 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:40.868875 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:40.868875 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:40.868875 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:40.868875 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:40.869714 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:40.869714 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:40.869714 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:40.869714 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:40.869714 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:40.869714 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:40.869714 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:40.869714 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:40.869714 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:40.869714 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:40.869714 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 04:15:40.869714 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:40.869714 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 04:15:40.869714 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:40.879936 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 474/945] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:15:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:40.880159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 472/946] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Aug 23 04:15:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:15:40.883202 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0b390-1c2b-4c84-8849-93e45978d930 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:40.884926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea5b9e-e821-424d-8c05-0377d6ab7791 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:40.889057 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:41.012702 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0b390-1c2b-4c84-8849-93e45978d930 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:41.015340 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea5b9e-e821-424d-8c05-0377d6ab7791 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:41.057464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5d9fa-eecf-45b6-90b8-dadf4368fee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:41.058260 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5d9fa-eecf-45b6-90b8-dadf4368fee4 None admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:41.058565 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5d9fa-eecf-45b6-90b8-dadf4368fee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:41.058907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5d9fa-eecf-45b6-90b8-dadf4368fee4 None admin] PATH_INFO: `/v3/projects/9f4373dfb5f9425eb73669e24d18a08f/users/d3538271ebfd41aabe9f3da85feb7bca/roles/3631d69e831a474b807aeca180b214dc` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:41.061128 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5d9fa-eecf-45b6-90b8-dadf4368fee4 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f4373dfb5f9425eb73669e24d18a08f, user_id=d3538271ebfd41aabe9f3da85feb7bca, role_id=3631d69e831a474b807aeca180b214dc)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:41.062149 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3706e-7689-4611-844f-341adb2b94ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:41.062717 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5d9fa-eecf-45b6-90b8-dadf4368fee4 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:41.063154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3706e-7689-4611-844f-341adb2b94ae None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:41.063596 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3706e-7689-4611-844f-341adb2b94ae None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:41.064053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3706e-7689-4611-844f-341adb2b94ae None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:41.064849 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3706e-7689-4611-844f-341adb2b94ae None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:41.065328 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3706e-7689-4611-844f-341adb2b94ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:41.066942 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3706e-7689-4611-844f-341adb2b94ae None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:41.077862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 473/947] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Aug 23 04:15:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:41.079148 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3706e-7689-4611-844f-341adb2b94ae None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:41.080585 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 475/948] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Aug 23 04:15:40 2022] PUT /identity/v3/projects/9f4373dfb5f9425eb73669e24d18a08f/users/d3538271ebfd41aabe9f3da85feb7bca/roles/3631d69e831a474b807aeca180b214dc => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:15:41.113110 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:41.197690 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3706e-7689-4611-844f-341adb2b94ae None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:41.238067 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7756f0-655c-4bbd-861d-d1600fb35cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:41.238986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7756f0-655c-4bbd-861d-d1600fb35cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:41.239526 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7756f0-655c-4bbd-861d-d1600fb35cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:41.239990 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7756f0-655c-4bbd-861d-d1600fb35cd4 None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:41.240850 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7756f0-655c-4bbd-861d-d1600fb35cd4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:41.241308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7756f0-655c-4bbd-861d-d1600fb35cd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:41.242849 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7756f0-655c-4bbd-861d-d1600fb35cd4 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:41.248941 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 474/949] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Aug 23 04:15:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:41.255595 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5d9fa-eecf-45b6-90b8-dadf4368fee4 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:41.293034 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779c650-5556-482b-8104-920301b13bf3 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:41.293541 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779c650-5556-482b-8104-920301b13bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:41.293541 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779c650-5556-482b-8104-920301b13bf3 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:41.295139 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 475/950] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:15:41.300061 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7e6b9-b536-4e85-a065-1779af1292aa None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:41.300559 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7e6b9-b536-4e85-a065-1779af1292aa None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:41.301012 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7e6b9-b536-4e85-a065-1779af1292aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:41.320051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-63e7e6b9-b536-4e85-a065-1779af1292aa None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:41.321580 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-63e7e6b9-b536-4e85-a065-1779af1292aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:41.326789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:41.370510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5d9fa-eecf-45b6-90b8-dadf4368fee4 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:41.393386 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 476/951] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 04:15:41.417112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b5a67-ecf5-4fbb-868d-c673dfdeeae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:41.418062 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b5a67-ecf5-4fbb-868d-c673dfdeeae6 None admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:41.418508 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b5a67-ecf5-4fbb-868d-c673dfdeeae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:41.418950 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b5a67-ecf5-4fbb-868d-c673dfdeeae6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:41.419781 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b5a67-ecf5-4fbb-868d-c673dfdeeae6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:41.421510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b5a67-ecf5-4fbb-868d-c673dfdeeae6 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:41.444919 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 476/952] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:15:41 2022] POST /identity/v3/endpoints => generated 349 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:15:41.672834 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e7e6b9-b536-4e85-a065-1779af1292aa None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:41.692251 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:41.776336 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e7e6b9-b536-4e85-a065-1779af1292aa None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:41.817711 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438467e9-c937-4d71-94dd-dc9ca3b03f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:41.818576 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438467e9-c937-4d71-94dd-dc9ca3b03f5d None admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:41.818851 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438467e9-c937-4d71-94dd-dc9ca3b03f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:41.819113 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438467e9-c937-4d71-94dd-dc9ca3b03f5d None admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:41.819756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438467e9-c937-4d71-94dd-dc9ca3b03f5d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:41.821220 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438467e9-c937-4d71-94dd-dc9ca3b03f5d None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:41.823838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: WARNING keystone.server.flask.common [None req-438467e9-c937-4d71-94dd-dc9ca3b03f5d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 23 04:15:41.843585 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.sql.core [None req-438467e9-c937-4d71-94dd-dc9ca3b03f5d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 23 04:15:41.843585 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 23 04:15:41.843585 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [parameters: {'id': '5a049cc2a4924a1394c773a349c35934', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 23 04:15:41.843585 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81934) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 23 04:15:41.860912 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-438467e9-c937-4d71-94dd-dc9ca3b03f5d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5a049cc2a4924a1394c773a349c35934.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5a049cc2a4924a1394c773a349c35934. Aug 23 04:15:41.860912 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:41.860912 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 04:15:41.860912 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 04:15:41.860912 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 04:15:41.860912 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 04:15:41.860912 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 04:15:41.860912 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 04:15:41.860912 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 04:15:41.860912 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application conn.query(q) Aug 23 04:15:41.860912 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 04:15:41.860912 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 04:15:41.860912 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 04:15:41.861759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application result.read() Aug 23 04:15:41.861759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 04:15:41.861759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 04:15:41.861759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 04:15:41.861759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 04:15:41.861759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 04:15:41.861759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 04:15:41.861759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 04:15:41.861759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 04:15:41.861759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 23 04:15:41.861759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:41.861759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 23 04:15:41.861759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:41.861759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:41.861759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 23 04:15:41.861759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 23 04:15:41.861759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 23 04:15:41.861759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 23 04:15:41.863247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 04:15:41.863247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application next(self.gen) Aug 23 04:15:41.863247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 23 04:15:41.863247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application yield resource Aug 23 04:15:41.863247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 04:15:41.863247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application next(self.gen) Aug 23 04:15:41.863247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 23 04:15:41.863247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self.session.rollback() Aug 23 04:15:41.863247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 23 04:15:41.863247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self.force_reraise() Aug 23 04:15:41.863247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 23 04:15:41.863247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise self.value Aug 23 04:15:41.863247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 23 04:15:41.863247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 23 04:15:41.863247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 23 04:15:41.863247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application session.commit() Aug 23 04:15:41.863247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 23 04:15:41.864271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 23 04:15:41.864271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 23 04:15:41.864271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self._prepare_impl() Aug 23 04:15:41.864271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 23 04:15:41.864271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self.session.flush() Aug 23 04:15:41.864271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 23 04:15:41.864271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self._flush(objects) Aug 23 04:15:41.864271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 23 04:15:41.864271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 23 04:15:41.864271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 23 04:15:41.864271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application compat.raise_( Aug 23 04:15:41.864271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 04:15:41.864271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception Aug 23 04:15:41.864271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 23 04:15:41.864271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application flush_context.execute() Aug 23 04:15:41.864271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 23 04:15:41.864271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rec.execute(self) Aug 23 04:15:41.865386 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 23 04:15:41.865386 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 23 04:15:41.865386 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 23 04:15:41.865386 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 23 04:15:41.865386 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 23 04:15:41.865386 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 23 04:15:41.865386 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 23 04:15:41.865386 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 23 04:15:41.865386 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 23 04:15:41.865386 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 23 04:15:41.865386 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 23 04:15:41.865386 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 23 04:15:41.865386 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 23 04:15:41.865386 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 23 04:15:41.865386 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 23 04:15:41.866185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 23 04:15:41.866185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 04:15:41.866185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception Aug 23 04:15:41.866185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 04:15:41.866185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 04:15:41.866185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 04:15:41.866185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 04:15:41.866185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 04:15:41.866185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 04:15:41.866185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 04:15:41.866185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application conn.query(q) Aug 23 04:15:41.866185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 04:15:41.866185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 04:15:41.866185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 04:15:41.866185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application result.read() Aug 23 04:15:41.866185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 04:15:41.866185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 04:15:41.867031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 04:15:41.867031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 04:15:41.867031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 04:15:41.867031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 04:15:41.867031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 04:15:41.867031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 04:15:41.867031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 23 04:15:41.867031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 23 04:15:41.867031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [parameters: {'id': '5a049cc2a4924a1394c773a349c35934', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 23 04:15:41.867031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 23 04:15:41.867031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:41.867031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:15:41.867031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:41.867031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:41.867031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 23 04:15:41.869869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 23 04:15:41.869869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 23 04:15:41.869869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 23 04:15:41.869869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 23 04:15:41.869869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:41.869869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:15:41.869869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:41.869869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:41.869869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:41.869869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:41.869869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:41.869869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:41.869869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:41.869869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:41.869869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:41.869869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:41.869869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:41.872344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:41.872344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 23 04:15:41.872344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 23 04:15:41.872344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:41.872344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:41.872344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 23 04:15:41.872344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 23 04:15:41.872344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5a049cc2a4924a1394c773a349c35934. Aug 23 04:15:41.872344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:41.872344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 477/953] 10.222.0.18 () {60 vars in 1140 bytes} [Tue Aug 23 04:15:41 2022] POST /identity/v3/projects => generated 262 bytes in 190 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:41.878922 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b5a67-ecf5-4fbb-868d-c673dfdeeae6 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:41.973714 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b5a67-ecf5-4fbb-868d-c673dfdeeae6 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:41.994785 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:42.009843 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3580a-1a5a-4583-aec7-c450d623a84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:42.010769 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3580a-1a5a-4583-aec7-c450d623a84c None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:42.011211 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3580a-1a5a-4583-aec7-c450d623a84c None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:42.011624 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3580a-1a5a-4583-aec7-c450d623a84c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:42.019203 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3580a-1a5a-4583-aec7-c450d623a84c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:42.021064 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3580a-1a5a-4583-aec7-c450d623a84c None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:42.032907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-9fb3580a-1a5a-4583-aec7-c450d623a84c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:42.032907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:42.032907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:42.032907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:42.032907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:42.032907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:42.032907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:42.032907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:42.032907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:42.032907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:42.032907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:42.032907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:42.032907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:15:42.032907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:15:42.032907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:15:42.034105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:15:42.034105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:42.034105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:42.034105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:42.034105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:42.034105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:42.034105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:42.034105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:42.034105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:42.034105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:42.034105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:42.034105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:42.034105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:42.034105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:42.034105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:42.034105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:42.034105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:42.035235 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:15:42.035235 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:15:42.035235 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:15:42.035235 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:15:42.035235 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:15:42.035235 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:15:42.035235 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:42.035235 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:42.037051 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 477/954] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:15:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:42.043642 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438467e9-c937-4d71-94dd-dc9ca3b03f5d None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:42.053974 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636bc94a-586e-4921-aabb-f15f07450d47 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:42.054184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636bc94a-586e-4921-aabb-f15f07450d47 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:42.054438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636bc94a-586e-4921-aabb-f15f07450d47 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:42.056063 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 478/955] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:15:42.061058 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0d06b-cc53-439f-ae92-f846ca06e84b None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:42.061350 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0d06b-cc53-439f-ae92-f846ca06e84b None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:42.061628 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0d06b-cc53-439f-ae92-f846ca06e84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:42.080066 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-3da0d06b-cc53-439f-ae92-f846ca06e84b None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:42.081348 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-3da0d06b-cc53-439f-ae92-f846ca06e84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:42.137159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:42.159594 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 479/956] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 04:15:42.171269 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438467e9-c937-4d71-94dd-dc9ca3b03f5d None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:42.206145 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c1f7a-8b54-4046-9f5a-b20e60a2c5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:42.206762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c1f7a-8b54-4046-9f5a-b20e60a2c5b9 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:42.207046 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c1f7a-8b54-4046-9f5a-b20e60a2c5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:42.207322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c1f7a-8b54-4046-9f5a-b20e60a2c5b9 None admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:42.208102 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c1f7a-8b54-4046-9f5a-b20e60a2c5b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:42.208422 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c1f7a-8b54-4046-9f5a-b20e60a2c5b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:42.210095 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c1f7a-8b54-4046-9f5a-b20e60a2c5b9 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:42.236131 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 478/957] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Aug 23 04:15:42 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:42.426745 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0d06b-cc53-439f-ae92-f846ca06e84b None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:42.531632 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0d06b-cc53-439f-ae92-f846ca06e84b None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:42.541465 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:42.569429 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b9edd9-72ee-47df-8df2-0b98258175bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:42.570325 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b9edd9-72ee-47df-8df2-0b98258175bd None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:42.570704 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b9edd9-72ee-47df-8df2-0b98258175bd None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:42.571074 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b9edd9-72ee-47df-8df2-0b98258175bd None admin] PATH_INFO: `/v3/services/compute` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:42.571779 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b9edd9-72ee-47df-8df2-0b98258175bd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:42.573316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b9edd9-72ee-47df-8df2-0b98258175bd None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:42.583208 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-93b9edd9-72ee-47df-8df2-0b98258175bd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 04:15:42.583208 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:42.583208 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 04:15:42.583208 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 04:15:42.583208 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 04:15:42.583208 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 04:15:42.583208 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 04:15:42.583208 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:42.583208 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 04:15:42.583208 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:42.583208 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:15:42.583208 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:42.583208 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:42.583208 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:42.583208 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:42.583208 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:42.583208 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:42.584620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:42.584620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:42.584620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:42.584620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:42.584620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:42.584620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:42.584620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 04:15:42.584620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 04:15:42.584620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 04:15:42.584620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 04:15:42.584620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:42.584620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:42.584620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:42.584620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:42.584620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:42.584620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:42.585626 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:42.585626 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:42.585626 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:42.585626 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:42.585626 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:42.585626 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:42.585626 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:42.585626 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:42.585626 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:42.585626 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:42.585626 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 04:15:42.585626 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:42.585626 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 04:15:42.585626 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:42.587535 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 480/958] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:15:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:42.594012 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c1f7a-8b54-4046-9f5a-b20e60a2c5b9 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:42.695728 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c1f7a-8b54-4046-9f5a-b20e60a2c5b9 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:42.725640 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:42.731831 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b53e7-d49d-4a6e-99d7-384d6faeff4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:42.732782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b53e7-d49d-4a6e-99d7-384d6faeff4f None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:42.733397 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b53e7-d49d-4a6e-99d7-384d6faeff4f None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:42.734041 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b53e7-d49d-4a6e-99d7-384d6faeff4f None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:42.734682 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62b53e7-d49d-4a6e-99d7-384d6faeff4f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:42.734984 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62b53e7-d49d-4a6e-99d7-384d6faeff4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:42.736407 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62b53e7-d49d-4a6e-99d7-384d6faeff4f None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:42.741718 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 479/959] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Aug 23 04:15:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:42.747376 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b9edd9-72ee-47df-8df2-0b98258175bd None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:42.855579 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b9edd9-72ee-47df-8df2-0b98258175bd None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:42.891041 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa0b55-102d-497a-91a8-738dd99b0c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:42.892478 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa0b55-102d-497a-91a8-738dd99b0c59 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:42.892983 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa0b55-102d-497a-91a8-738dd99b0c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:42.893464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa0b55-102d-497a-91a8-738dd99b0c59 None admin] PATH_INFO: `/v3/services` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:42.894283 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa0b55-102d-497a-91a8-738dd99b0c59 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:42.894789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa0b55-102d-497a-91a8-738dd99b0c59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:42.896586 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa0b55-102d-497a-91a8-738dd99b0c59 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:42.901890 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 481/960] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Aug 23 04:15:42 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:43.118874 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632eb53-952e-4ff0-84a6-c72b47d064d8 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:43.119186 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632eb53-952e-4ff0-84a6-c72b47d064d8 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:43.119387 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632eb53-952e-4ff0-84a6-c72b47d064d8 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:43.121040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 480/961] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:43.126636 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c4fff-219c-4f7e-80f3-e5ec0b14c1d5 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:43.126930 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c4fff-219c-4f7e-80f3-e5ec0b14c1d5 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:43.127207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c4fff-219c-4f7e-80f3-e5ec0b14c1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:43.145407 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-8c7c4fff-219c-4f7e-80f3-e5ec0b14c1d5 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:43.146869 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-8c7c4fff-219c-4f7e-80f3-e5ec0b14c1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:43.177239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:43.217818 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 482/962] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:15:43.452234 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c65400-5fa2-452f-acb2-b31bc246f793 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:43.452571 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c65400-5fa2-452f-acb2-b31bc246f793 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:43.452803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c65400-5fa2-452f-acb2-b31bc246f793 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:43.454175 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 481/963] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:43.468429 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dcc3a3-dad5-49ea-8151-3b26b7cd4c75 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:43.468742 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dcc3a3-dad5-49ea-8151-3b26b7cd4c75 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:43.468972 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dcc3a3-dad5-49ea-8151-3b26b7cd4c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:43.489568 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-67dcc3a3-dad5-49ea-8151-3b26b7cd4c75 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:43.490768 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-67dcc3a3-dad5-49ea-8151-3b26b7cd4c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:43.500876 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c65400-5fa2-452f-acb2-b31bc246f793 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:43.576013 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 483/964] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:15:43.616927 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c65400-5fa2-452f-acb2-b31bc246f793 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:43.653630 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f51aa-d302-4ba8-9430-22abb7e458aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:43.654312 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f51aa-d302-4ba8-9430-22abb7e458aa None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:43.654593 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f51aa-d302-4ba8-9430-22abb7e458aa None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:43.654861 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f51aa-d302-4ba8-9430-22abb7e458aa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:43.665894 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f51aa-d302-4ba8-9430-22abb7e458aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:15:43.671308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-788f51aa-d302-4ba8-9430-22abb7e458aa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 04:15:43.671308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:43.671308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:43.671308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:43.671308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:43.671308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:43.671308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:43.671308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:43.671308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:43.671308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:43.671308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:43.671308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:43.671308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:15:43.671308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:15:43.671308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:15:43.671308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:15:43.672218 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:15:43.672218 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:15:43.672218 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:15:43.672218 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:15:43.672218 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:43.672218 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:43.672218 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:43.672218 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:43.672218 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:43.672218 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:43.672218 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:43.672218 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:43.672218 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:43.672218 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:43.672218 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:43.672218 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:43.673044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:43.673044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:43.673044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:43.673044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:43.673044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:15:43.673044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:15:43.673044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:15:43.673044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:15:43.673044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:15:43.673044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:15:43.673044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 04:15:43.673044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:43.673044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 482/965] 10.222.0.18 () {56 vars in 1120 bytes} [Tue Aug 23 04:15:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:15:43.678866 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dcc3a3-dad5-49ea-8151-3b26b7cd4c75 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:43.772914 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:43.778429 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dcc3a3-dad5-49ea-8151-3b26b7cd4c75 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:43.813999 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5499e0-e700-4d09-9bb4-6b91f0f5a15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:43.814615 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5499e0-e700-4d09-9bb4-6b91f0f5a15b None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:43.814893 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5499e0-e700-4d09-9bb4-6b91f0f5a15b None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:43.815167 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5499e0-e700-4d09-9bb4-6b91f0f5a15b None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:43.815837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5499e0-e700-4d09-9bb4-6b91f0f5a15b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:43.816251 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5499e0-e700-4d09-9bb4-6b91f0f5a15b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:43.817676 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5499e0-e700-4d09-9bb4-6b91f0f5a15b None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:43.827284 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 484/966] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Aug 23 04:15:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:43.833173 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f51aa-d302-4ba8-9430-22abb7e458aa None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:43.874633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5499e0-e700-4d09-9bb4-6b91f0f5a15b None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:43.899414 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:43.945748 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f51aa-d302-4ba8-9430-22abb7e458aa None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:43.981172 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5499e0-e700-4d09-9bb4-6b91f0f5a15b None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:43.990655 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b2b36-e898-42d7-99a2-f28419a84958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:43.991318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b2b36-e898-42d7-99a2-f28419a84958 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:43.991890 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b2b36-e898-42d7-99a2-f28419a84958 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:43.992196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b2b36-e898-42d7-99a2-f28419a84958 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:44.003705 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b2b36-e898-42d7-99a2-f28419a84958 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:44.005534 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b2b36-e898-42d7-99a2-f28419a84958 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:44.015451 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf1dea-4281-4ebf-a543-607b2e7efd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:44.016195 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf1dea-4281-4ebf-a543-607b2e7efd97 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:44.016474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf1dea-4281-4ebf-a543-607b2e7efd97 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:44.016746 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf1dea-4281-4ebf-a543-607b2e7efd97 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:44.017337 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf1dea-4281-4ebf-a543-607b2e7efd97 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:44.018710 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf1dea-4281-4ebf-a543-607b2e7efd97 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:44.021376 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-ff8b2b36-e898-42d7-99a2-f28419a84958 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:44.021376 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:44.021376 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:44.021376 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:44.021376 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:44.021376 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:44.021376 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:44.021376 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:44.021376 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:44.021376 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:44.021376 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:44.021376 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:44.021376 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:44.021376 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:44.021376 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:44.021376 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:44.022295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:44.022295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:44.022295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:44.022295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:44.022295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:44.022295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:44.022295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:44.022295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:44.022295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:44.022295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:44.022295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:44.022295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:44.022295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:44.022295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:44.022295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:44.022295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:44.022295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:44.023220 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:44.023220 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:44.023220 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:44.023220 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 483/967] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:44.028053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-9fcf1dea-4281-4ebf-a543-607b2e7efd97 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 04:15:44.028053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:44.028053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 04:15:44.028053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 04:15:44.028053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 04:15:44.028053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 04:15:44.028053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 04:15:44.028053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:44.028053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 04:15:44.028053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:44.028053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:15:44.028053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:44.028053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:44.028053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:44.028053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:44.028053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:44.028053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:44.028988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:44.028988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:44.028988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:44.028988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:44.028988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:44.028988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:44.028988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 04:15:44.028988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 04:15:44.028988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 04:15:44.028988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 04:15:44.028988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:44.028988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:44.028988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:44.028988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:44.028988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:44.028988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:44.029789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:44.029789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:44.029789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:44.029789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:44.029789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:44.029789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:44.029789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:44.029789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:44.029789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:44.029789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:44.029789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 04:15:44.029789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:44.029789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 04:15:44.029789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:44.031386 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b2b36-e898-42d7-99a2-f28419a84958 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:44.031526 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 485/968] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Aug 23 04:15:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:15:44.037237 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf1dea-4281-4ebf-a543-607b2e7efd97 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:44.071796 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:44.154362 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b2b36-e898-42d7-99a2-f28419a84958 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:44.162342 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf1dea-4281-4ebf-a543-607b2e7efd97 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:44.164863 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:44.203118 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac136d-f446-469f-92c9-3e0dfc837b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:44.203943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac136d-f446-469f-92c9-3e0dfc837b1d None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:44.204325 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac136d-f446-469f-92c9-3e0dfc837b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:44.204842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac136d-f446-469f-92c9-3e0dfc837b1d None admin] PATH_INFO: `/v3/services` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:44.205480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ca210-f60b-420c-aafc-763020479659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:44.205689 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac136d-f446-469f-92c9-3e0dfc837b1d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:44.206088 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac136d-f446-469f-92c9-3e0dfc837b1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:44.206552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ca210-f60b-420c-aafc-763020479659 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:44.206970 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ca210-f60b-420c-aafc-763020479659 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:44.207377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ca210-f60b-420c-aafc-763020479659 None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:44.208054 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac136d-f446-469f-92c9-3e0dfc837b1d None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:44.208426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ca210-f60b-420c-aafc-763020479659 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:44.208870 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ca210-f60b-420c-aafc-763020479659 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:44.211059 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ca210-f60b-420c-aafc-763020479659 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:44.218840 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 486/969] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Aug 23 04:15:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:44.219797 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac136d-f446-469f-92c9-3e0dfc837b1d None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:44.229933 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 484/970] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:44.234950 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ca210-f60b-420c-aafc-763020479659 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:44.330842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:44.342049 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ca210-f60b-420c-aafc-763020479659 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:44.353169 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac136d-f446-469f-92c9-3e0dfc837b1d None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:44.379110 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf637441-c992-4772-81cf-c682dfdb3f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:44.379752 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf637441-c992-4772-81cf-c682dfdb3f04 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:44.380105 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf637441-c992-4772-81cf-c682dfdb3f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:44.380382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf637441-c992-4772-81cf-c682dfdb3f04 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:44.387171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70044f2-2da3-46c0-86aa-eb20d19ee020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:44.387779 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70044f2-2da3-46c0-86aa-eb20d19ee020 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:44.388098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70044f2-2da3-46c0-86aa-eb20d19ee020 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:44.388368 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70044f2-2da3-46c0-86aa-eb20d19ee020 None admin] PATH_INFO: `/v3/services` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:44.388999 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70044f2-2da3-46c0-86aa-eb20d19ee020 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:44.389296 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70044f2-2da3-46c0-86aa-eb20d19ee020 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:44.390214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf637441-c992-4772-81cf-c682dfdb3f04 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:44.390654 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70044f2-2da3-46c0-86aa-eb20d19ee020 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:44.391781 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf637441-c992-4772-81cf-c682dfdb3f04 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:44.400812 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 487/971] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Aug 23 04:15:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:44.400812 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75881fe0-d07b-480e-b8ca-be22caae7749 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:44.400812 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75881fe0-d07b-480e-b8ca-be22caae7749 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:44.400812 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75881fe0-d07b-480e-b8ca-be22caae7749 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:44.400812 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 488/972] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:44.401430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75881fe0-d07b-480e-b8ca-be22caae7749 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:44.407508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-cf637441-c992-4772-81cf-c682dfdb3f04 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 04:15:44.407508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:44.407508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:44.407508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:44.407508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:44.407508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:44.407508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:44.407508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:44.407508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:44.407508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:44.407508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:44.407508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:44.407508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:15:44.407508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:15:44.407508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:15:44.407508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:15:44.408667 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:44.408667 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:44.408667 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:15:44.408667 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:44.408667 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:15:44.408667 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:44.408667 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:44.408667 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:44.408667 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:44.408667 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:44.408667 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:44.408667 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:44.408667 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:44.408667 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:44.408667 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:44.408667 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:44.408667 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:44.408667 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:44.409616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:44.409616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:44.409616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:15:44.409616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:15:44.409616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:15:44.409616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:15:44.409616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:15:44.409616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:15:44.409616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 04:15:44.409616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:44.409616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 485/973] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Aug 23 04:15:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:15:44.410229 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544a227-aa4f-4c2c-9242-99c7e2956164 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:44.410452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544a227-aa4f-4c2c-9242-99c7e2956164 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:44.410728 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544a227-aa4f-4c2c-9242-99c7e2956164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:44.430027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-a544a227-aa4f-4c2c-9242-99c7e2956164 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:44.431553 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-a544a227-aa4f-4c2c-9242-99c7e2956164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:44.461646 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:44.513743 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 486/974] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:15:44.514417 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544a227-aa4f-4c2c-9242-99c7e2956164 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:44.529128 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75881fe0-d07b-480e-b8ca-be22caae7749 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:44.574411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b7e15-8453-4203-bf39-e3181d97141d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:44.575682 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b7e15-8453-4203-bf39-e3181d97141d None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:44.576422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b7e15-8453-4203-bf39-e3181d97141d None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:44.576858 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b7e15-8453-4203-bf39-e3181d97141d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:44.577969 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b7e15-8453-4203-bf39-e3181d97141d None admin] RBAC: Adding query filter params (interface=admin, service_id=3bc2958100e04ff3b7239b5ea9bdf191, region_id=RegionOne) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:44.578416 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b7e15-8453-4203-bf39-e3181d97141d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:44.580592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b7e15-8453-4203-bf39-e3181d97141d None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:44.588869 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 489/975] 10.222.0.18 () {56 vars in 1249 bytes} [Tue Aug 23 04:15:44 2022] GET /identity/v3/endpoints?service_id=3bc2958100e04ff3b7239b5ea9bdf191&interface=admin®ion_id=RegionOne => generated 193 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:44.594172 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b7e15-8453-4203-bf39-e3181d97141d None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:44.611172 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:44.635119 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544a227-aa4f-4c2c-9242-99c7e2956164 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:44.677045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f100d1f-25e0-4237-82bf-9f5e0cfe7f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:44.678232 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f100d1f-25e0-4237-82bf-9f5e0cfe7f49 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:44.678866 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f100d1f-25e0-4237-82bf-9f5e0cfe7f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:44.679487 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f100d1f-25e0-4237-82bf-9f5e0cfe7f49 None admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:44.681344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f100d1f-25e0-4237-82bf-9f5e0cfe7f49 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:44.681859 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f100d1f-25e0-4237-82bf-9f5e0cfe7f49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:44.683844 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f100d1f-25e0-4237-82bf-9f5e0cfe7f49 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:44.704876 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 487/976] 10.222.0.18 () {56 vars in 1149 bytes} [Tue Aug 23 04:15:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:44.715935 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b7e15-8453-4203-bf39-e3181d97141d None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:44.716145 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f100d1f-25e0-4237-82bf-9f5e0cfe7f49 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:44.748156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:44.753803 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae1adc-bf79-458a-bd58-9c79df6a0fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:44.754560 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae1adc-bf79-458a-bd58-9c79df6a0fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:44.755121 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae1adc-bf79-458a-bd58-9c79df6a0fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:44.755430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae1adc-bf79-458a-bd58-9c79df6a0fa1 None admin] PATH_INFO: `/v3/services` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:44.756199 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae1adc-bf79-458a-bd58-9c79df6a0fa1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:44.757678 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae1adc-bf79-458a-bd58-9c79df6a0fa1 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:44.772280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 490/977] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Aug 23 04:15:44 2022] GET /identity/v3/services => generated 1497 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:15:44.804021 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae1adc-bf79-458a-bd58-9c79df6a0fa1 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:44.844032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f100d1f-25e0-4237-82bf-9f5e0cfe7f49 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:44.888539 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be9486-ea77-4c7c-aa4d-250b5adbc87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:44.889178 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be9486-ea77-4c7c-aa4d-250b5adbc87e None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:44.889473 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be9486-ea77-4c7c-aa4d-250b5adbc87e None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:44.889759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be9486-ea77-4c7c-aa4d-250b5adbc87e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:44.903684 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:44.927852 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae1adc-bf79-458a-bd58-9c79df6a0fa1 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:44.966293 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be9486-ea77-4c7c-aa4d-250b5adbc87e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:44.966495 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f598ff75-8606-4214-bfa4-fd57732f8294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:44.966930 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598ff75-8606-4214-bfa4-fd57732f8294 None admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:44.967197 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598ff75-8606-4214-bfa4-fd57732f8294 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:44.967461 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598ff75-8606-4214-bfa4-fd57732f8294 None admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:44.967766 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be9486-ea77-4c7c-aa4d-250b5adbc87e None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:44.968119 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f598ff75-8606-4214-bfa4-fd57732f8294 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:44.969523 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f598ff75-8606-4214-bfa4-fd57732f8294 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:44.972102 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: WARNING keystone.server.flask.common [None req-f598ff75-8606-4214-bfa4-fd57732f8294 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 23 04:15:44.984021 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:44.986446 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.sql.core [None req-f598ff75-8606-4214-bfa4-fd57732f8294 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 23 04:15:44.986446 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 23 04:15:44.986446 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [parameters: {'user_id': '5fdfbee261e4481da7a286b2f5887fc5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 23 04:15:44.986446 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81933) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 23 04:15:44.998183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-f598ff75-8606-4214-bfa4-fd57732f8294 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 23 04:15:44.998183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:44.998183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 04:15:44.998183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 04:15:44.998183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 04:15:44.998183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 04:15:44.998183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 04:15:44.998183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 04:15:44.998183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 04:15:44.998183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application conn.query(q) Aug 23 04:15:44.998183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 04:15:44.998183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 04:15:44.998183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 04:15:44.998183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application result.read() Aug 23 04:15:44.998183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 04:15:44.999055 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 04:15:44.999055 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 04:15:44.999055 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 04:15:44.999055 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 04:15:44.999055 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 04:15:44.999055 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 04:15:44.999055 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 04:15:44.999055 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 23 04:15:44.999055 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:44.999055 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 23 04:15:44.999055 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:44.999055 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:44.999055 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 23 04:15:44.999055 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 23 04:15:44.999055 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 23 04:15:44.999055 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 23 04:15:44.999055 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 04:15:44.999055 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application next(self.gen) Aug 23 04:15:45.000092 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 23 04:15:45.000092 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application yield resource Aug 23 04:15:45.000092 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 04:15:45.000092 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application next(self.gen) Aug 23 04:15:45.000092 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 23 04:15:45.000092 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self.session.rollback() Aug 23 04:15:45.000092 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 23 04:15:45.000092 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self.force_reraise() Aug 23 04:15:45.000092 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 23 04:15:45.000092 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise self.value Aug 23 04:15:45.000092 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 23 04:15:45.000092 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 23 04:15:45.000092 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 23 04:15:45.000092 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application session.commit() Aug 23 04:15:45.000092 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 23 04:15:45.000092 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 23 04:15:45.000092 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 23 04:15:45.001040 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self._prepare_impl() Aug 23 04:15:45.001040 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 23 04:15:45.001040 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self.session.flush() Aug 23 04:15:45.001040 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 23 04:15:45.001040 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self._flush(objects) Aug 23 04:15:45.001040 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 23 04:15:45.001040 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 23 04:15:45.001040 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 23 04:15:45.001040 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application compat.raise_( Aug 23 04:15:45.001040 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 04:15:45.001040 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception Aug 23 04:15:45.001040 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 23 04:15:45.001040 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application flush_context.execute() Aug 23 04:15:45.001040 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 23 04:15:45.001040 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rec.execute(self) Aug 23 04:15:45.001040 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 23 04:15:45.001040 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 23 04:15:45.002770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 23 04:15:45.002770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 23 04:15:45.002770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 23 04:15:45.002770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 23 04:15:45.002770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 23 04:15:45.002770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 23 04:15:45.002770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 23 04:15:45.002770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 23 04:15:45.002770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 23 04:15:45.002770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 23 04:15:45.002770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 23 04:15:45.002770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 23 04:15:45.002770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 23 04:15:45.002770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 23 04:15:45.002770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 04:15:45.002770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception Aug 23 04:15:45.004520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 04:15:45.004520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 04:15:45.004520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 04:15:45.004520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 04:15:45.004520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 04:15:45.004520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 04:15:45.004520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 04:15:45.004520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application conn.query(q) Aug 23 04:15:45.004520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 04:15:45.004520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 04:15:45.004520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 04:15:45.004520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application result.read() Aug 23 04:15:45.004520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 04:15:45.004520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 04:15:45.004520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 04:15:45.004520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 04:15:45.006708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 04:15:45.006708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 04:15:45.006708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 04:15:45.006708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 04:15:45.006708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 23 04:15:45.006708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 23 04:15:45.006708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [parameters: {'user_id': '5fdfbee261e4481da7a286b2f5887fc5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 23 04:15:45.006708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 23 04:15:45.006708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:45.006708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:15:45.006708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:45.006708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:45.006708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:45.006708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:45.006708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:45.008844 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:45.008844 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:45.008844 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:45.008844 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:45.008844 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:45.008844 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:45.008844 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:45.008844 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 23 04:15:45.008844 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 23 04:15:45.008844 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:45.008844 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:45.008844 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:15:45.008844 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:45.008844 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:15:45.008844 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:45.008844 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 23 04:15:45.008844 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 23 04:15:45.009758 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 23 04:15:45.009758 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 23 04:15:45.009758 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 23 04:15:45.009758 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 23 04:15:45.009758 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 23 04:15:45.009758 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:45.009758 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 491/978] 10.222.0.18 () {60 vars in 1131 bytes} [Tue Aug 23 04:15:44 2022] POST /identity/v3/users => generated 160 bytes in 198 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:45.009758 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb887c6-8a74-487e-aa41-33e3e3196fe4 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:45.009758 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb887c6-8a74-487e-aa41-33e3e3196fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:45.009758 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb887c6-8a74-487e-aa41-33e3e3196fe4 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:45.010368 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 492/979] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:45.012817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb887c6-8a74-487e-aa41-33e3e3196fe4 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:45.092232 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 488/980] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:15:44 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 04:15:45.094383 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e971f6-4a40-49a8-8701-f89f6d313176 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:45.094608 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e971f6-4a40-49a8-8701-f89f6d313176 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:45.094874 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e971f6-4a40-49a8-8701-f89f6d313176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:45.113039 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-e9e971f6-4a40-49a8-8701-f89f6d313176 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:45.114252 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-e9e971f6-4a40-49a8-8701-f89f6d313176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:45.121729 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb887c6-8a74-487e-aa41-33e3e3196fe4 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:45.158883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f18659-94a9-4345-8f0b-1e9bb69f1d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:45.159512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f18659-94a9-4345-8f0b-1e9bb69f1d72 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:45.159801 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f18659-94a9-4345-8f0b-1e9bb69f1d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:45.160106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f18659-94a9-4345-8f0b-1e9bb69f1d72 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:45.168557 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f18659-94a9-4345-8f0b-1e9bb69f1d72 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:45.170211 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f18659-94a9-4345-8f0b-1e9bb69f1d72 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:45.174165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:45.183609 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 489/981] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:15:45.184133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-91f18659-94a9-4345-8f0b-1e9bb69f1d72 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 04:15:45.184133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:45.184133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:45.184133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:45.184133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:45.184133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:45.184133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:45.184133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:45.184133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:45.184133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:45.184133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:45.184133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:45.184133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:15:45.184133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:15:45.184133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:15:45.184133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:15:45.185027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:45.185027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:45.185027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:15:45.185027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:45.185027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:15:45.185027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:45.185027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:45.185027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:45.185027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:45.185027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:45.185027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:45.185027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:45.185027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:45.185027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:45.185027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:45.185027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:45.185027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:45.185027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:45.185933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:45.185933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:45.185933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:15:45.185933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:15:45.185933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:15:45.185933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:15:45.185933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:15:45.185933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:15:45.185933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 04:15:45.185933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:45.187661 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e971f6-4a40-49a8-8701-f89f6d313176 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:45.190002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 493/982] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:15:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:45.190855 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f18659-94a9-4345-8f0b-1e9bb69f1d72 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:45.245565 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:45.308557 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f18659-94a9-4345-8f0b-1e9bb69f1d72 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:45.322743 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e971f6-4a40-49a8-8701-f89f6d313176 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:45.344868 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7afe9-af50-4a04-aea3-1698381bf113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:45.345399 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7afe9-af50-4a04-aea3-1698381bf113 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:45.345666 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7afe9-af50-4a04-aea3-1698381bf113 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:45.345932 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7afe9-af50-4a04-aea3-1698381bf113 None admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:45.346650 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7afe9-af50-4a04-aea3-1698381bf113 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:45.346921 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7afe9-af50-4a04-aea3-1698381bf113 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:45.348381 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7afe9-af50-4a04-aea3-1698381bf113 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:45.367077 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf04e32-f3fe-434e-87c9-d098c7f7e4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:45.368108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf04e32-f3fe-434e-87c9-d098c7f7e4cb None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:45.368635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf04e32-f3fe-434e-87c9-d098c7f7e4cb None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:45.369101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf04e32-f3fe-434e-87c9-d098c7f7e4cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:45.382393 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf04e32-f3fe-434e-87c9-d098c7f7e4cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:45.383851 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf04e32-f3fe-434e-87c9-d098c7f7e4cb None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:45.392457 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 494/983] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Aug 23 04:15:45 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:45.400556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-6bf04e32-f3fe-434e-87c9-d098c7f7e4cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:45.400556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:45.400556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:45.400556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:45.400556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:45.400556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:45.400556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:45.400556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:45.400556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:45.400556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:45.400556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:45.400556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:45.400556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:45.400556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:45.400556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:45.400556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:45.401552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:45.401552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:45.401552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:45.401552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:45.401552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:45.401552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:45.401552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:45.401552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:45.401552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:45.401552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:45.401552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:45.401552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:45.401552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:45.401552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:45.401552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:45.401552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:45.401552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:45.402575 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:45.402575 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:45.402575 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:45.404968 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 490/984] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:45.411172 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7afe9-af50-4a04-aea3-1698381bf113 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:45.474936 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf04e32-f3fe-434e-87c9-d098c7f7e4cb None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:45.490168 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:45.550049 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7afe9-af50-4a04-aea3-1698381bf113 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:45.587097 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:45.593644 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7901ee-8570-4611-ad15-ac7dcc5a44a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:45.594330 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7901ee-8570-4611-ad15-ac7dcc5a44a9 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:45.595841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7901ee-8570-4611-ad15-ac7dcc5a44a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:45.595841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7901ee-8570-4611-ad15-ac7dcc5a44a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:45.595841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7901ee-8570-4611-ad15-ac7dcc5a44a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:45.595841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7901ee-8570-4611-ad15-ac7dcc5a44a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:45.596531 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf04e32-f3fe-434e-87c9-d098c7f7e4cb None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:45.597129 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7901ee-8570-4611-ad15-ac7dcc5a44a9 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:45.609958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 495/985] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:45.616608 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7901ee-8570-4611-ad15-ac7dcc5a44a9 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:45.644229 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c0252-5f4f-4eba-96bd-e30260a43d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:45.644994 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c0252-5f4f-4eba-96bd-e30260a43d3a None admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:45.645365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c0252-5f4f-4eba-96bd-e30260a43d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:45.645771 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c0252-5f4f-4eba-96bd-e30260a43d3a None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:45.646712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3c0252-5f4f-4eba-96bd-e30260a43d3a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:45.648246 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3c0252-5f4f-4eba-96bd-e30260a43d3a None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:45.671254 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 491/986] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Aug 23 04:15:45 2022] POST /identity/v3/services => generated 242 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:15:45.745291 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7901ee-8570-4611-ad15-ac7dcc5a44a9 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:45.765413 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:45.793844 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b761c-d80a-4875-b237-ccb56273f297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:45.794576 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b761c-d80a-4875-b237-ccb56273f297 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:45.794857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b761c-d80a-4875-b237-ccb56273f297 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:45.795126 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b761c-d80a-4875-b237-ccb56273f297 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:45.803296 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b761c-d80a-4875-b237-ccb56273f297 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:45.804947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b761c-d80a-4875-b237-ccb56273f297 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:45.817934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-cc9b761c-d80a-4875-b237-ccb56273f297 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:45.817934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:45.817934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:45.817934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:45.817934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:45.817934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:45.817934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:45.817934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:45.817934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:45.817934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:45.817934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:45.817934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:45.817934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:15:45.817934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:15:45.817934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:15:45.818953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:15:45.818953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:45.818953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:45.818953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:45.818953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:45.818953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:45.818953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:45.818953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:45.818953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:45.818953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:45.818953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:45.818953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:45.818953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:45.818953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:45.818953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:45.818953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:45.818953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:45.825027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:15:45.825027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:15:45.825027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:15:45.825027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:15:45.825027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:15:45.825027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:15:45.825027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:45.825027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:45.825027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 496/987] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:15:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:45.827365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c0252-5f4f-4eba-96bd-e30260a43d3a None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:45.937778 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c0252-5f4f-4eba-96bd-e30260a43d3a None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:45.972079 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97396412-d5b8-4a15-8a97-8b4e1d78a573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:45.972740 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97396412-d5b8-4a15-8a97-8b4e1d78a573 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:45.973020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97396412-d5b8-4a15-8a97-8b4e1d78a573 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:45.973287 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97396412-d5b8-4a15-8a97-8b4e1d78a573 None admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:45.974324 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97396412-d5b8-4a15-8a97-8b4e1d78a573 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:45.974629 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97396412-d5b8-4a15-8a97-8b4e1d78a573 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:45.976091 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97396412-d5b8-4a15-8a97-8b4e1d78a573 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:45.992871 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 492/988] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:15:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:45.994552 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b761c-d80a-4875-b237-ccb56273f297 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:46.099162 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b761c-d80a-4875-b237-ccb56273f297 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:46.133215 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a6f56-6ea3-445d-bd3d-3bf9f0fb40c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:46.133823 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a6f56-6ea3-445d-bd3d-3bf9f0fb40c8 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:46.134099 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a6f56-6ea3-445d-bd3d-3bf9f0fb40c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:46.134368 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a6f56-6ea3-445d-bd3d-3bf9f0fb40c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:46.135095 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283a6f56-6ea3-445d-bd3d-3bf9f0fb40c8 None admin] RBAC: Adding query filter params (role.id=3631d69e831a474b807aeca180b214dc, scope.project.id=9f4373dfb5f9425eb73669e24d18a08f, user.id=d3538271ebfd41aabe9f3da85feb7bca) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:46.135399 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283a6f56-6ea3-445d-bd3d-3bf9f0fb40c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:46.136844 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283a6f56-6ea3-445d-bd3d-3bf9f0fb40c8 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:46.143673 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 497/989] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:15:45 2022] GET /identity/v3/role_assignments?user.id=d3538271ebfd41aabe9f3da85feb7bca&scope.project.id=9f4373dfb5f9425eb73669e24d18a08f&role.id=3631d69e831a474b807aeca180b214dc => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:46.695234 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8fa32-5bbe-490b-9512-b0abf5f10cd6 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:46.695809 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8fa32-5bbe-490b-9512-b0abf5f10cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:46.695809 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8fa32-5bbe-490b-9512-b0abf5f10cd6 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:46.697337 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 493/990] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:46.702439 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6cf42e-6c98-4eda-a4eb-45774e784dcd None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:46.702723 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6cf42e-6c98-4eda-a4eb-45774e784dcd None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:46.703005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6cf42e-6c98-4eda-a4eb-45774e784dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:46.721974 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-2f6cf42e-6c98-4eda-a4eb-45774e784dcd None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:46.723397 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-2f6cf42e-6c98-4eda-a4eb-45774e784dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:46.753192 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:46.796282 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 498/991] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:46 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:15:47.068579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8fa32-5bbe-490b-9512-b0abf5f10cd6 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:47.088472 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:47.181314 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8fa32-5bbe-490b-9512-b0abf5f10cd6 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:47.218688 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa936f9-ff34-4d05-88e1-85c279d484bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:47.219333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa936f9-ff34-4d05-88e1-85c279d484bd None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:47.219609 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa936f9-ff34-4d05-88e1-85c279d484bd None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:47.219881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa936f9-ff34-4d05-88e1-85c279d484bd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:47.220500 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa936f9-ff34-4d05-88e1-85c279d484bd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:47.221873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa936f9-ff34-4d05-88e1-85c279d484bd None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:47.231709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-1fa936f9-ff34-4d05-88e1-85c279d484bd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 04:15:47.231709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:47.231709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 04:15:47.231709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 04:15:47.231709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 04:15:47.231709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 04:15:47.231709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 04:15:47.231709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:47.231709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 04:15:47.231709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:47.231709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:15:47.231709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:47.231709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:47.231709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:47.231709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:47.231709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:47.231709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:47.232823 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:47.232823 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:47.232823 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:47.232823 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:47.232823 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:47.232823 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:47.232823 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 04:15:47.232823 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 04:15:47.232823 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 04:15:47.232823 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 04:15:47.232823 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:47.232823 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:47.232823 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:47.232823 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:47.232823 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:47.232823 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:47.233654 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:47.233654 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:47.233654 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:47.233654 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:47.233654 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:47.233654 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:47.233654 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:47.233654 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:47.233654 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:47.233654 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:47.233654 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 04:15:47.233654 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:47.233654 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 04:15:47.233654 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:47.236407 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 494/992] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Aug 23 04:15:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:47.238936 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6cf42e-6c98-4eda-a4eb-45774e784dcd None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:47.345770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6cf42e-6c98-4eda-a4eb-45774e784dcd None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:47.380087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7080e-48e6-4355-87ce-25778891943c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:47.380837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7080e-48e6-4355-87ce-25778891943c None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:47.381116 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7080e-48e6-4355-87ce-25778891943c None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:47.381386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7080e-48e6-4355-87ce-25778891943c None admin] PATH_INFO: `/v3/services` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:47.382069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7080e-48e6-4355-87ce-25778891943c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:47.382373 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7080e-48e6-4355-87ce-25778891943c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:47.383756 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7080e-48e6-4355-87ce-25778891943c None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:47.393602 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 499/993] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Aug 23 04:15:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:47.394841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa936f9-ff34-4d05-88e1-85c279d484bd None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:47.486899 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:47.508419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa936f9-ff34-4d05-88e1-85c279d484bd None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:47.543133 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd9b22-0d6d-4517-94ea-81063910e23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:47.543806 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd9b22-0d6d-4517-94ea-81063910e23a None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:47.544109 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd9b22-0d6d-4517-94ea-81063910e23a None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:47.544392 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd9b22-0d6d-4517-94ea-81063910e23a None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:47.545006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcd9b22-0d6d-4517-94ea-81063910e23a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:47.545303 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcd9b22-0d6d-4517-94ea-81063910e23a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:47.546673 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcd9b22-0d6d-4517-94ea-81063910e23a None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:47.551951 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 495/994] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Aug 23 04:15:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:47.557722 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7080e-48e6-4355-87ce-25778891943c None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:47.590877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:47.597447 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfaed53-824c-41fe-8917-4e323cf0c4a0 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:47.597673 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfaed53-824c-41fe-8917-4e323cf0c4a0 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:47.597948 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfaed53-824c-41fe-8917-4e323cf0c4a0 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:47.599772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 496/995] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:47.604834 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f4560-ff9f-42b4-ab51-2c796e02880a None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:47.605118 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f4560-ff9f-42b4-ab51-2c796e02880a None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:47.605390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f4560-ff9f-42b4-ab51-2c796e02880a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:47.623840 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-862f4560-ff9f-42b4-ab51-2c796e02880a None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:47.625080 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-862f4560-ff9f-42b4-ab51-2c796e02880a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:47.673746 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7080e-48e6-4355-87ce-25778891943c None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:47.709909 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9d8d14-f053-40f6-a718-31b2eedff960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:47.710533 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d8d14-f053-40f6-a718-31b2eedff960 None admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:47.710810 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d8d14-f053-40f6-a718-31b2eedff960 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:47.711062 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d8d14-f053-40f6-a718-31b2eedff960 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:47.711759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9d8d14-f053-40f6-a718-31b2eedff960 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:47.712250 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 497/996] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:47 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:15:47.713479 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9d8d14-f053-40f6-a718-31b2eedff960 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:47.731277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 500/997] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:15:47 2022] POST /identity/v3/endpoints => generated 325 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:15:47.972581 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f4560-ff9f-42b4-ab51-2c796e02880a None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:48.062292 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:48.092424 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f4560-ff9f-42b4-ab51-2c796e02880a None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:48.104062 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e1daa-6646-4d86-815d-de34b499d418 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:48.104375 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e1daa-6646-4d86-815d-de34b499d418 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:48.104591 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e1daa-6646-4d86-815d-de34b499d418 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:48.105966 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 501/998] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:15:48.110737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71d550-fb6a-49d0-ab53-7a8b9a92fd0e None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:48.111017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71d550-fb6a-49d0-ab53-7a8b9a92fd0e None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:48.111279 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71d550-fb6a-49d0-ab53-7a8b9a92fd0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:48.130082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-4a71d550-fb6a-49d0-ab53-7a8b9a92fd0e None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:48.131400 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-4a71d550-fb6a-49d0-ab53-7a8b9a92fd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:48.136927 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfcb146-5c31-497f-af2f-cb8fa88dae80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:48.137782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcb146-5c31-497f-af2f-cb8fa88dae80 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:48.138230 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcb146-5c31-497f-af2f-cb8fa88dae80 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:48.138704 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcb146-5c31-497f-af2f-cb8fa88dae80 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:48.139853 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfcb146-5c31-497f-af2f-cb8fa88dae80 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:48.141758 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfcb146-5c31-497f-af2f-cb8fa88dae80 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:48.151812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-6dfcb146-5c31-497f-af2f-cb8fa88dae80 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 04:15:48.151812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:48.151812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 04:15:48.151812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 04:15:48.151812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 04:15:48.151812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 04:15:48.151812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 04:15:48.151812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:48.151812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 04:15:48.151812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:48.151812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:15:48.151812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:48.151812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:48.151812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:48.151812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:48.151812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:48.151812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:48.152838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:48.152838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:48.152838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:48.152838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:48.152838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:48.152838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:48.152838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 04:15:48.152838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 04:15:48.152838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 04:15:48.152838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 04:15:48.152838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:48.152838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:48.152838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:48.152838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:48.152838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:48.152838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:48.153730 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:48.153730 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:48.153730 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:48.153730 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:48.153730 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:48.153730 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:48.153730 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:48.153730 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:48.153730 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:48.153730 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:48.153730 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 04:15:48.153730 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:48.153730 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 04:15:48.153730 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:48.156066 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 498/999] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:15:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:48.160461 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfcb146-5c31-497f-af2f-cb8fa88dae80 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:48.207980 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 502/1000] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:48 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 04:15:48.265160 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfcb146-5c31-497f-af2f-cb8fa88dae80 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:48.299545 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06d173-11b8-44e2-ae6b-1d8bad014261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:48.300175 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06d173-11b8-44e2-ae6b-1d8bad014261 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:48.300431 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06d173-11b8-44e2-ae6b-1d8bad014261 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:48.300725 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06d173-11b8-44e2-ae6b-1d8bad014261 None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:48.301350 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06d173-11b8-44e2-ae6b-1d8bad014261 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:48.301744 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06d173-11b8-44e2-ae6b-1d8bad014261 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:48.303623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06d173-11b8-44e2-ae6b-1d8bad014261 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:48.304726 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:48.311029 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 499/1001] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Aug 23 04:15:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:48.316409 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71d550-fb6a-49d0-ab53-7a8b9a92fd0e None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:48.342065 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:48.428265 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71d550-fb6a-49d0-ab53-7a8b9a92fd0e None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:48.466904 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9283d-060c-4193-acfd-e655042727ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:48.467529 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9283d-060c-4193-acfd-e655042727ae None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:48.467807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9283d-060c-4193-acfd-e655042727ae None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:48.468107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9283d-060c-4193-acfd-e655042727ae None admin] PATH_INFO: `/v3/services` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:48.468753 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9283d-060c-4193-acfd-e655042727ae None admin] RBAC: Adding query filter params (type=compute) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:48.469056 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9283d-060c-4193-acfd-e655042727ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:48.470422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9283d-060c-4193-acfd-e655042727ae None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:48.474983 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06d173-11b8-44e2-ae6b-1d8bad014261 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:48.480509 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 503/1002] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Aug 23 04:15:48 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:48.481354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9283d-060c-4193-acfd-e655042727ae None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:48.570268 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:48.579390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06d173-11b8-44e2-ae6b-1d8bad014261 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:48.586426 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9283d-060c-4193-acfd-e655042727ae None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:48.615460 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106cc70-f5fd-4c7e-9ce0-c626ab8446f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:48.616083 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106cc70-f5fd-4c7e-9ce0-c626ab8446f5 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:48.616363 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106cc70-f5fd-4c7e-9ce0-c626ab8446f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:48.616632 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106cc70-f5fd-4c7e-9ce0-c626ab8446f5 None admin] PATH_INFO: `/v3/services/image` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:48.617215 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9106cc70-f5fd-4c7e-9ce0-c626ab8446f5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:48.618608 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9106cc70-f5fd-4c7e-9ce0-c626ab8446f5 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:48.624018 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95975e-70a2-4481-9b30-3fed086e0dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:48.624633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95975e-70a2-4481-9b30-3fed086e0dfb None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:48.624907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95975e-70a2-4481-9b30-3fed086e0dfb None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:48.625174 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95975e-70a2-4481-9b30-3fed086e0dfb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:48.625870 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf95975e-70a2-4481-9b30-3fed086e0dfb None admin] RBAC: Adding query filter params (interface=public, service_id=ca92c80bf8ca4a23a5fac5b910f4b073, region_id=RegionOne) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:48.626194 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf95975e-70a2-4481-9b30-3fed086e0dfb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:48.627477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-9106cc70-f5fd-4c7e-9ce0-c626ab8446f5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 04:15:48.627477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:48.627477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 04:15:48.627477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 04:15:48.627477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 04:15:48.627477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 04:15:48.627477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 04:15:48.627477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:48.627477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 04:15:48.627477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:48.627477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:15:48.627477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:48.627477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:48.627477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:48.627477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:48.627477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:48.627477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:48.628420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:48.628420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:48.628420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:48.628420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:48.628420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:48.628420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:48.628420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 04:15:48.628420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 04:15:48.628420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 04:15:48.628420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 04:15:48.628420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:48.628420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:48.628420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:48.628420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:48.628420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:48.628420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:48.632839 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:48.632839 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:48.632839 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:48.632839 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:48.632839 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:48.632839 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:48.632839 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:48.632839 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:48.632839 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:48.632839 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:48.632839 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 04:15:48.632839 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:48.632839 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 04:15:48.632839 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:48.633854 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf95975e-70a2-4481-9b30-3fed086e0dfb None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:48.633929 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 500/1003] 10.222.0.18 () {56 vars in 1105 bytes} [Tue Aug 23 04:15:48 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:15:48.635014 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106cc70-f5fd-4c7e-9ce0-c626ab8446f5 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:48.640277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 504/1004] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Aug 23 04:15:48 2022] GET /identity/v3/endpoints?service_id=ca92c80bf8ca4a23a5fac5b910f4b073&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:48.641835 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95975e-70a2-4481-9b30-3fed086e0dfb None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:48.702081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:48.742576 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106cc70-f5fd-4c7e-9ce0-c626ab8446f5 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:48.746404 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95975e-70a2-4481-9b30-3fed086e0dfb None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:48.777217 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7827e5e5-9ccd-4571-8fab-b932bdbc51c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:48.777827 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827e5e5-9ccd-4571-8fab-b932bdbc51c5 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:48.778103 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827e5e5-9ccd-4571-8fab-b932bdbc51c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:48.778367 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827e5e5-9ccd-4571-8fab-b932bdbc51c5 None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:48.778993 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7827e5e5-9ccd-4571-8fab-b932bdbc51c5 None admin] RBAC: Adding query filter params (name=image) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:48.779291 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7827e5e5-9ccd-4571-8fab-b932bdbc51c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:48.780691 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7827e5e5-9ccd-4571-8fab-b932bdbc51c5 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:48.783293 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cffa57-bd4d-46f0-a506-9bed53d1cc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:48.784159 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cffa57-bd4d-46f0-a506-9bed53d1cc6b None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:48.784613 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cffa57-bd4d-46f0-a506-9bed53d1cc6b None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:48.785056 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cffa57-bd4d-46f0-a506-9bed53d1cc6b None admin] PATH_INFO: `/v3/services` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:48.785367 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 501/1005] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:48.786197 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cffa57-bd4d-46f0-a506-9bed53d1cc6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:48.787482 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cffa57-bd4d-46f0-a506-9bed53d1cc6b None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:48.791783 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7827e5e5-9ccd-4571-8fab-b932bdbc51c5 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:48.797516 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 505/1006] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Aug 23 04:15:48 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:15:48.919438 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7827e5e5-9ccd-4571-8fab-b932bdbc51c5 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:48.949196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:48.958852 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bcebe-3c9d-4440-b9f9-2cc77a050a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:48.959645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bcebe-3c9d-4440-b9f9-2cc77a050a5c None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:48.960140 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bcebe-3c9d-4440-b9f9-2cc77a050a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:48.960446 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bcebe-3c9d-4440-b9f9-2cc77a050a5c None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:48.961138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bcebe-3c9d-4440-b9f9-2cc77a050a5c None admin] RBAC: Adding query filter params (type=image) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:48.961898 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bcebe-3c9d-4440-b9f9-2cc77a050a5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:48.962814 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bcebe-3c9d-4440-b9f9-2cc77a050a5c None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:48.968065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 502/1007] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:48 2022] GET /identity/v3/services?type=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:49.631162 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e3a11-0ba0-427b-9ad1-706f0e8568d1 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:49.631706 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e3a11-0ba0-427b-9ad1-706f0e8568d1 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:49.631706 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e3a11-0ba0-427b-9ad1-706f0e8568d1 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:49.633258 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 506/1008] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:49.638323 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7061b00-8415-4f26-8d36-b5702302fc90 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:49.638571 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7061b00-8415-4f26-8d36-b5702302fc90 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:49.638872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7061b00-8415-4f26-8d36-b5702302fc90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:49.657269 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-e7061b00-8415-4f26-8d36-b5702302fc90 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:49.658445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-e7061b00-8415-4f26-8d36-b5702302fc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:49.726315 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 503/1009] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:49 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:15:49.989864 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e3a11-0ba0-427b-9ad1-706f0e8568d1 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:50.006211 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:50.098413 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e3a11-0ba0-427b-9ad1-706f0e8568d1 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:50.132368 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da58ff7-8db6-4718-8a9b-cd4722c298c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:50.133001 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da58ff7-8db6-4718-8a9b-cd4722c298c4 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:50.133254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da58ff7-8db6-4718-8a9b-cd4722c298c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:50.133536 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da58ff7-8db6-4718-8a9b-cd4722c298c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:50.134634 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da58ff7-8db6-4718-8a9b-cd4722c298c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:50.136087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da58ff7-8db6-4718-8a9b-cd4722c298c4 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:50.139942 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 507/1010] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:50.143395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7061b00-8415-4f26-8d36-b5702302fc90 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:50.241918 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7061b00-8415-4f26-8d36-b5702302fc90 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:50.261748 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:50.277330 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1a002-362f-41c3-ac1b-a54b3fad95d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:50.277934 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1a002-362f-41c3-ac1b-a54b3fad95d2 None admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:50.278203 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1a002-362f-41c3-ac1b-a54b3fad95d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:50.278467 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1a002-362f-41c3-ac1b-a54b3fad95d2 None admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:50.279110 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa1a002-362f-41c3-ac1b-a54b3fad95d2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:50.280583 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa1a002-362f-41c3-ac1b-a54b3fad95d2 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:50.316012 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 504/1011] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Aug 23 04:15:50 2022] POST /identity/v3/projects => generated 311 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:15:50.716426 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68b285-aced-4df2-bf0b-1dc46d5ff10c None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:50.716998 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68b285-aced-4df2-bf0b-1dc46d5ff10c None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:50.716998 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68b285-aced-4df2-bf0b-1dc46d5ff10c None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:50.718514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 508/1012] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:50.723789 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747eb7c-46e4-4ef1-94e0-9d9a6668df37 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:50.724236 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747eb7c-46e4-4ef1-94e0-9d9a6668df37 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:50.724493 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747eb7c-46e4-4ef1-94e0-9d9a6668df37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:50.742945 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-1747eb7c-46e4-4ef1-94e0-9d9a6668df37 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:50.744186 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-1747eb7c-46e4-4ef1-94e0-9d9a6668df37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:50.814531 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 505/1013] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:50 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:15:50.930182 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9fb30-dbe1-4ae5-828e-e090fc65d2f4 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:50.930482 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9fb30-dbe1-4ae5-828e-e090fc65d2f4 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:50.930896 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9fb30-dbe1-4ae5-828e-e090fc65d2f4 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:50.935961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 509/1014] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:50.937941 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96038de3-7410-4150-b842-8db13512835a None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:50.938234 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96038de3-7410-4150-b842-8db13512835a None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:50.938508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96038de3-7410-4150-b842-8db13512835a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:50.957510 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-96038de3-7410-4150-b842-8db13512835a None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:50.959032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-96038de3-7410-4150-b842-8db13512835a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:51.033265 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 506/1015] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:50 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:15:51.083745 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d9fb30-dbe1-4ae5-828e-e090fc65d2f4 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:51.168674 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:51.189186 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d9fb30-dbe1-4ae5-828e-e090fc65d2f4 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:51.224524 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e492a8-18d3-4109-8a35-269bf076369e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:51.225159 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e492a8-18d3-4109-8a35-269bf076369e None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:51.225439 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e492a8-18d3-4109-8a35-269bf076369e None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:51.225708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e492a8-18d3-4109-8a35-269bf076369e None admin] PATH_INFO: `/v3/services/compute` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:51.226287 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e492a8-18d3-4109-8a35-269bf076369e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:51.227645 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e492a8-18d3-4109-8a35-269bf076369e None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:51.236212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-53e492a8-18d3-4109-8a35-269bf076369e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 04:15:51.236212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:51.236212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 04:15:51.236212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 04:15:51.236212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 04:15:51.236212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 04:15:51.236212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 04:15:51.236212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:51.236212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 04:15:51.236212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:51.236212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:15:51.236212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:51.236212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:51.236212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:51.236212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:51.236212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:51.236212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:51.237146 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:51.237146 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:51.237146 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:51.237146 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:51.237146 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:51.237146 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:51.237146 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 04:15:51.237146 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 04:15:51.237146 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 04:15:51.237146 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 04:15:51.237146 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:51.237146 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:51.237146 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:51.237146 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:51.237146 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:51.237146 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:51.237953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:51.237953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:51.237953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:51.237953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:51.237953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:51.237953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:51.237953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:51.237953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:51.237953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:51.237953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:51.237953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 04:15:51.237953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:51.237953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 04:15:51.237953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:51.237953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 510/1016] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:15:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:51.245228 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96038de3-7410-4150-b842-8db13512835a None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:51.277224 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:51.295440 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e492a8-18d3-4109-8a35-269bf076369e None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:51.354853 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96038de3-7410-4150-b842-8db13512835a None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:51.392249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9a7f6-d38f-497f-8420-8b8bf645afb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:51.392864 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9a7f6-d38f-497f-8420-8b8bf645afb4 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:51.393146 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9a7f6-d38f-497f-8420-8b8bf645afb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:51.393420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9a7f6-d38f-497f-8420-8b8bf645afb4 None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:51.394057 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d9a7f6-d38f-497f-8420-8b8bf645afb4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:51.394360 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d9a7f6-d38f-497f-8420-8b8bf645afb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:51.395829 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d9a7f6-d38f-497f-8420-8b8bf645afb4 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:51.401171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 507/1017] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Aug 23 04:15:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:51.401888 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e492a8-18d3-4109-8a35-269bf076369e None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:51.406651 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9a7f6-d38f-497f-8420-8b8bf645afb4 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:51.447951 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee9612-532d-48d9-9f06-02047cd0d2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:51.448570 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee9612-532d-48d9-9f06-02047cd0d2f6 None admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:51.448848 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee9612-532d-48d9-9f06-02047cd0d2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:51.449112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee9612-532d-48d9-9f06-02047cd0d2f6 None admin] PATH_INFO: `/v3/services` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:51.449770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee9612-532d-48d9-9f06-02047cd0d2f6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:51.451145 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee9612-532d-48d9-9f06-02047cd0d2f6 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:51.472608 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 511/1018] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Aug 23 04:15:51 2022] POST /identity/v3/services => generated 242 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:15:51.518959 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:51.526977 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9a7f6-d38f-497f-8420-8b8bf645afb4 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:51.570854 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2600007d-854c-4c0f-8c24-21632da5007a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:51.571483 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600007d-854c-4c0f-8c24-21632da5007a None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:51.571831 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600007d-854c-4c0f-8c24-21632da5007a None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:51.572197 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600007d-854c-4c0f-8c24-21632da5007a None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:51.572906 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2600007d-854c-4c0f-8c24-21632da5007a None admin] RBAC: Adding query filter params (type=compute) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:51.573279 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2600007d-854c-4c0f-8c24-21632da5007a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:51.574779 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2600007d-854c-4c0f-8c24-21632da5007a None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:51.580686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 508/1019] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Aug 23 04:15:51 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:51.586539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee9612-532d-48d9-9f06-02047cd0d2f6 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:51.597640 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:51.697808 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee9612-532d-48d9-9f06-02047cd0d2f6 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:51.733751 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eae51c-4aa2-4afe-b042-52985ca1fa43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:51.735188 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eae51c-4aa2-4afe-b042-52985ca1fa43 None admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:51.735670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eae51c-4aa2-4afe-b042-52985ca1fa43 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:51.736241 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eae51c-4aa2-4afe-b042-52985ca1fa43 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:51.737140 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eae51c-4aa2-4afe-b042-52985ca1fa43 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:51.738711 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eae51c-4aa2-4afe-b042-52985ca1fa43 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:51.762474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 512/1020] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:15:51 2022] POST /identity/v3/endpoints => generated 334 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:15:52.248933 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79af695-bce3-4ccd-9f0f-c37c2ede8adf None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:52.249487 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79af695-bce3-4ccd-9f0f-c37c2ede8adf None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:52.249487 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79af695-bce3-4ccd-9f0f-c37c2ede8adf None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:52.250777 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 509/1021] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:52.256567 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aab106-578a-47d3-80d7-fddd4cf0d9ac None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:52.256954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aab106-578a-47d3-80d7-fddd4cf0d9ac None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:52.257300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aab106-578a-47d3-80d7-fddd4cf0d9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:52.276207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-21aab106-578a-47d3-80d7-fddd4cf0d9ac None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:52.277418 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-21aab106-578a-47d3-80d7-fddd4cf0d9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:52.339536 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:52.359042 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 513/1022] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:15:52.634246 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79af695-bce3-4ccd-9f0f-c37c2ede8adf None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:52.737829 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79af695-bce3-4ccd-9f0f-c37c2ede8adf None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:52.771766 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793451f3-7dd5-4194-9b68-6cdbb5ac84a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:52.772428 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793451f3-7dd5-4194-9b68-6cdbb5ac84a4 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:52.772708 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793451f3-7dd5-4194-9b68-6cdbb5ac84a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:52.772977 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793451f3-7dd5-4194-9b68-6cdbb5ac84a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:52.774102 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793451f3-7dd5-4194-9b68-6cdbb5ac84a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:52.775545 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793451f3-7dd5-4194-9b68-6cdbb5ac84a4 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:52.777325 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 510/1023] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:52.782980 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aab106-578a-47d3-80d7-fddd4cf0d9ac None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:52.895607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aab106-578a-47d3-80d7-fddd4cf0d9ac None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:52.933681 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c969e-dbdd-4e44-bd72-1e416e407459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:52.934724 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c969e-dbdd-4e44-bd72-1e416e407459 None admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:52.935198 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c969e-dbdd-4e44-bd72-1e416e407459 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:52.935659 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c969e-dbdd-4e44-bd72-1e416e407459 None admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:52.936571 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c969e-dbdd-4e44-bd72-1e416e407459 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:52.938231 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c969e-dbdd-4e44-bd72-1e416e407459 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:52.967948 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 514/1024] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Aug 23 04:15:52 2022] POST /identity/v3/users => generated 285 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:15:53.405140 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e1f7fa-d209-484a-8583-521b3f4a50a3 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:53.405709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e1f7fa-d209-484a-8583-521b3f4a50a3 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:53.405709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e1f7fa-d209-484a-8583-521b3f4a50a3 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:53.407169 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 511/1025] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:53.412570 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785a412-0476-48fb-8acb-51703a46df03 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:53.412821 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785a412-0476-48fb-8acb-51703a46df03 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:53.413117 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785a412-0476-48fb-8acb-51703a46df03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:53.431724 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-7785a412-0476-48fb-8acb-51703a46df03 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:53.432950 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-7785a412-0476-48fb-8acb-51703a46df03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:53.481786 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:53.515961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 515/1026] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:15:53.765965 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75320c-1f66-46de-a6e4-d22507e2c170 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:53.766156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75320c-1f66-46de-a6e4-d22507e2c170 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:53.766456 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75320c-1f66-46de-a6e4-d22507e2c170 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:53.767763 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 512/1027] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:53.773354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f90013-b4d5-4648-a0cf-cfb2664ad823 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:53.773647 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f90013-b4d5-4648-a0cf-cfb2664ad823 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:53.773889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f90013-b4d5-4648-a0cf-cfb2664ad823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:53.777113 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75320c-1f66-46de-a6e4-d22507e2c170 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:53.796751 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-b9f90013-b4d5-4648-a0cf-cfb2664ad823 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:53.798351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-b9f90013-b4d5-4648-a0cf-cfb2664ad823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:53.804393 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:53.881392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 516/1028] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:15:53.890053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75320c-1f66-46de-a6e4-d22507e2c170 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:53.924610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d193992e-e552-4a6c-86e9-3acfbf97124d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:53.925248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193992e-e552-4a6c-86e9-3acfbf97124d None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:53.925528 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193992e-e552-4a6c-86e9-3acfbf97124d None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:53.925805 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193992e-e552-4a6c-86e9-3acfbf97124d None admin] PATH_INFO: `/v3/services/image` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:53.926383 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d193992e-e552-4a6c-86e9-3acfbf97124d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:53.927941 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d193992e-e552-4a6c-86e9-3acfbf97124d None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:53.936756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-d193992e-e552-4a6c-86e9-3acfbf97124d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 04:15:53.936756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:53.936756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 04:15:53.936756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 04:15:53.936756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 04:15:53.936756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 04:15:53.936756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 04:15:53.936756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:53.936756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 04:15:53.936756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:53.936756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:15:53.936756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:53.936756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:53.936756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:53.936756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:53.936756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:53.936756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:53.937762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:53.937762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:53.937762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:53.937762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:53.937762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:53.937762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:53.937762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 04:15:53.937762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 04:15:53.937762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 04:15:53.937762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 04:15:53.937762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:53.937762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:53.937762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:53.937762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:53.937762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:53.937762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:53.941020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:53.941020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:53.941020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:53.941020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:53.941020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:53.941020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:53.941020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:53.941020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:53.941020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:53.941020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:53.941020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 04:15:53.941020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:53.941020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 04:15:53.941020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:53.941020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 513/1029] 10.222.0.18 () {56 vars in 1105 bytes} [Tue Aug 23 04:15:53 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:15:53.944470 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f90013-b4d5-4648-a0cf-cfb2664ad823 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:54.049928 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f90013-b4d5-4648-a0cf-cfb2664ad823 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:54.052611 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:54.090612 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79cf8d-e0f7-419a-af90-1143e91070c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:54.091549 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79cf8d-e0f7-419a-af90-1143e91070c9 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:54.092069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79cf8d-e0f7-419a-af90-1143e91070c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:54.092535 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79cf8d-e0f7-419a-af90-1143e91070c9 None admin] PATH_INFO: `/v3/services` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:54.093354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79cf8d-e0f7-419a-af90-1143e91070c9 None admin] RBAC: Adding query filter params (name=image) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:54.093832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79cf8d-e0f7-419a-af90-1143e91070c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:54.095453 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79cf8d-e0f7-419a-af90-1143e91070c9 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:54.101268 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 517/1030] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:54.106756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d193992e-e552-4a6c-86e9-3acfbf97124d None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:54.142566 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79cf8d-e0f7-419a-af90-1143e91070c9 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:54.199034 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:54.213902 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d193992e-e552-4a6c-86e9-3acfbf97124d None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:54.243314 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79cf8d-e0f7-419a-af90-1143e91070c9 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:54.248129 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b8625-d5df-4de2-b442-a1e63ae04fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:54.248957 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b8625-d5df-4de2-b442-a1e63ae04fbd None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:54.249384 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b8625-d5df-4de2-b442-a1e63ae04fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:54.249826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b8625-d5df-4de2-b442-a1e63ae04fbd None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:54.250629 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b8625-d5df-4de2-b442-a1e63ae04fbd None admin] RBAC: Adding query filter params (type=image) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:54.250938 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b8625-d5df-4de2-b442-a1e63ae04fbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:54.252357 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b8625-d5df-4de2-b442-a1e63ae04fbd None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:54.262083 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 514/1031] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:54 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:54.263117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b8625-d5df-4de2-b442-a1e63ae04fbd None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:54.283342 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db37e9e-38f7-4de2-b4d0-8d5491ea9c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:54.283986 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db37e9e-38f7-4de2-b4d0-8d5491ea9c73 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:54.284269 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db37e9e-38f7-4de2-b4d0-8d5491ea9c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:54.284538 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db37e9e-38f7-4de2-b4d0-8d5491ea9c73 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:54.294663 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db37e9e-38f7-4de2-b4d0-8d5491ea9c73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:15:54.299271 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-1db37e9e-38f7-4de2-b4d0-8d5491ea9c73 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 04:15:54.299271 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:54.299271 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:54.299271 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:54.299271 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:54.299271 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:54.299271 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:54.299271 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:54.299271 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:54.299271 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:54.299271 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:54.299271 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:54.299271 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:15:54.299271 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:15:54.299271 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:15:54.299271 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:15:54.300474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:15:54.300474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:15:54.300474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:15:54.300474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:15:54.300474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:54.300474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:54.300474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:54.300474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:54.300474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:54.300474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:54.300474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:54.300474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:54.300474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:54.300474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:54.300474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:54.300474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:54.301592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:54.301592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:54.301592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:54.301592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:54.301592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:15:54.301592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:15:54.301592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:15:54.301592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:15:54.301592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:15:54.301592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:15:54.301592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 04:15:54.301592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:54.305333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 518/1032] 10.222.0.18 () {56 vars in 1120 bytes} [Tue Aug 23 04:15:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:54.306167 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db37e9e-38f7-4de2-b4d0-8d5491ea9c73 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:54.335415 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:54.377015 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b8625-d5df-4de2-b442-a1e63ae04fbd None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:54.410411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db37e9e-38f7-4de2-b4d0-8d5491ea9c73 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:54.414273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4823b544-1fac-43c3-8fc9-1918d12fb22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:54.414897 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823b544-1fac-43c3-8fc9-1918d12fb22d None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:54.415174 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823b544-1fac-43c3-8fc9-1918d12fb22d None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:54.415450 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823b544-1fac-43c3-8fc9-1918d12fb22d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:54.416186 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4823b544-1fac-43c3-8fc9-1918d12fb22d None admin] RBAC: Adding query filter params (interface=public, service_id=90ba046c71ea42a8b01bc0cb4259a129, region_id=RegionOne) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:54.416495 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4823b544-1fac-43c3-8fc9-1918d12fb22d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:54.417896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4823b544-1fac-43c3-8fc9-1918d12fb22d None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:54.423782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 515/1033] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Aug 23 04:15:54 2022] GET /identity/v3/endpoints?service_id=90ba046c71ea42a8b01bc0cb4259a129&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:54.429531 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4823b544-1fac-43c3-8fc9-1918d12fb22d None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:54.448461 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:54.448892 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0144eb6-319b-425b-bd45-3edb221cb6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:54.449509 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0144eb6-319b-425b-bd45-3edb221cb6be None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:54.450191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0144eb6-319b-425b-bd45-3edb221cb6be None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:54.450562 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0144eb6-319b-425b-bd45-3edb221cb6be None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:54.451268 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0144eb6-319b-425b-bd45-3edb221cb6be None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:54.451634 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0144eb6-319b-425b-bd45-3edb221cb6be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:54.453318 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0144eb6-319b-425b-bd45-3edb221cb6be None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:54.467946 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 519/1034] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Aug 23 04:15:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:54.473336 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0144eb6-319b-425b-bd45-3edb221cb6be None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:54.550012 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4823b544-1fac-43c3-8fc9-1918d12fb22d None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:54.571073 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:54.582432 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0144eb6-319b-425b-bd45-3edb221cb6be None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:54.585445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36bc99-c6f2-4281-b3ac-0e0b0265f710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:54.585762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36bc99-c6f2-4281-b3ac-0e0b0265f710 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:54.586040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36bc99-c6f2-4281-b3ac-0e0b0265f710 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:54.586307 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36bc99-c6f2-4281-b3ac-0e0b0265f710 None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:54.586915 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e36bc99-c6f2-4281-b3ac-0e0b0265f710 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:54.588741 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e36bc99-c6f2-4281-b3ac-0e0b0265f710 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:54.603949 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 516/1035] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Aug 23 04:15:54 2022] GET /identity/v3/services => generated 1957 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:15:54.631558 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f3b1b-bbbc-4e16-9e6b-b1fff74a333c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:54.632348 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f3b1b-bbbc-4e16-9e6b-b1fff74a333c None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:54.632685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f3b1b-bbbc-4e16-9e6b-b1fff74a333c None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:54.633133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f3b1b-bbbc-4e16-9e6b-b1fff74a333c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:54.645739 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f3b1b-bbbc-4e16-9e6b-b1fff74a333c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:54.647276 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f3b1b-bbbc-4e16-9e6b-b1fff74a333c None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:54.663960 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-3c0f3b1b-bbbc-4e16-9e6b-b1fff74a333c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:54.663960 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:54.663960 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:54.663960 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:54.663960 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:54.663960 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:54.663960 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:54.663960 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:54.663960 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:54.663960 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:54.663960 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:54.663960 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:54.663960 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:54.663960 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:54.663960 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:54.663960 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:54.664988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:54.664988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:54.664988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:54.664988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:54.664988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:54.664988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:54.664988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:54.664988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:54.664988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:54.664988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:54.664988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:54.664988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:54.664988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:54.664988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:54.664988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:54.664988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:54.664988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:54.665857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:54.665857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:54.665857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:54.668317 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 520/1036] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:15:54.671284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36bc99-c6f2-4281-b3ac-0e0b0265f710 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:54.764295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:54.784208 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36bc99-c6f2-4281-b3ac-0e0b0265f710 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:54.817702 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25ad54-55e2-4214-83c4-335327b601d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:54.818370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25ad54-55e2-4214-83c4-335327b601d4 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:54.818644 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25ad54-55e2-4214-83c4-335327b601d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:54.818909 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25ad54-55e2-4214-83c4-335327b601d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:54.819552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25ad54-55e2-4214-83c4-335327b601d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:54.819848 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25ad54-55e2-4214-83c4-335327b601d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:54.821456 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25ad54-55e2-4214-83c4-335327b601d4 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:54.833574 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 517/1037] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:54.839846 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f3b1b-bbbc-4e16-9e6b-b1fff74a333c None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:54.917347 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f362c7-1035-4c81-8f6a-3ea64ad8dc8a None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:54.917614 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f362c7-1035-4c81-8f6a-3ea64ad8dc8a None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:54.917890 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f362c7-1035-4c81-8f6a-3ea64ad8dc8a None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:54.919257 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 518/1038] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:15:54.924563 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cbb2e-28b3-48cd-8f9a-428ca30389b0 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:54.924851 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cbb2e-28b3-48cd-8f9a-428ca30389b0 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:54.925130 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cbb2e-28b3-48cd-8f9a-428ca30389b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:54.944013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-605cbb2e-28b3-48cd-8f9a-428ca30389b0 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:54.947817 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-605cbb2e-28b3-48cd-8f9a-428ca30389b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:54.957226 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f3b1b-bbbc-4e16-9e6b-b1fff74a333c None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:54.996497 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17e285-3d95-4b14-9015-f0dd35f2442f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:54.997138 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17e285-3d95-4b14-9015-f0dd35f2442f None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:54.997410 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17e285-3d95-4b14-9015-f0dd35f2442f None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:54.997678 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17e285-3d95-4b14-9015-f0dd35f2442f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:55.005985 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f17e285-3d95-4b14-9015-f0dd35f2442f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:55.007399 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f17e285-3d95-4b14-9015-f0dd35f2442f None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:55.019280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-2f17e285-3d95-4b14-9015-f0dd35f2442f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 04:15:55.019280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:55.019280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:55.019280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:55.019280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:55.019280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:55.019280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:55.019280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:55.019280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:55.019280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:55.019280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:55.019280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:55.019280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:15:55.019280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:15:55.019280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:15:55.019280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:15:55.020380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:55.020380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:55.020380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:15:55.020380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:55.020380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:15:55.020380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:55.020380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:55.020380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:55.020380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:55.020380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:55.020380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:55.020380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:55.020380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:55.020380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:55.020380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:55.020380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:55.020380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:55.020380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:55.021325 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:55.021325 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:55.021325 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:15:55.021325 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:15:55.021325 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:15:55.021325 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:15:55.021325 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:15:55.021325 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:15:55.021325 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 04:15:55.021325 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:55.021325 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 521/1039] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Aug 23 04:15:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:55.026701 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17e285-3d95-4b14-9015-f0dd35f2442f None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:55.028019 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 519/1040] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:54 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 04:15:55.029503 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:55.131219 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17e285-3d95-4b14-9015-f0dd35f2442f None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:55.164624 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025179a-2e4a-4325-b120-e6303abad914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:55.165227 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025179a-2e4a-4325-b120-e6303abad914 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:55.165503 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025179a-2e4a-4325-b120-e6303abad914 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:55.165771 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025179a-2e4a-4325-b120-e6303abad914 None admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:55.166493 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025179a-2e4a-4325-b120-e6303abad914 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:55.166793 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025179a-2e4a-4325-b120-e6303abad914 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:55.168234 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025179a-2e4a-4325-b120-e6303abad914 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:55.185660 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 522/1041] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Aug 23 04:15:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:55.191469 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605cbb2e-28b3-48cd-8f9a-428ca30389b0 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:55.265056 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:55.292473 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025179a-2e4a-4325-b120-e6303abad914 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:55.307796 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605cbb2e-28b3-48cd-8f9a-428ca30389b0 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:55.346260 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e29078b-f41e-4cb1-a022-4b9160b7f25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:55.346880 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29078b-f41e-4cb1-a022-4b9160b7f25d None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:55.347160 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29078b-f41e-4cb1-a022-4b9160b7f25d None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:55.347432 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29078b-f41e-4cb1-a022-4b9160b7f25d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:55.354675 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:55.395476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025179a-2e4a-4325-b120-e6303abad914 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:55.418296 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e29078b-f41e-4cb1-a022-4b9160b7f25d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:55.419782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e29078b-f41e-4cb1-a022-4b9160b7f25d None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:55.430171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ccbab2-7062-4f5b-a72d-60bbc4a1bc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:55.431056 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccbab2-7062-4f5b-a72d-60bbc4a1bc1d None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:55.431473 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccbab2-7062-4f5b-a72d-60bbc4a1bc1d None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:55.431881 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccbab2-7062-4f5b-a72d-60bbc4a1bc1d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:55.442204 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ccbab2-7062-4f5b-a72d-60bbc4a1bc1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:15:55.447053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-31ccbab2-7062-4f5b-a72d-60bbc4a1bc1d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:15:55.447053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:55.447053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:55.447053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:55.447053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:55.447053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:55.447053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:55.447053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:55.447053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:55.447053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:55.447053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:55.447053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:55.447053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:15:55.447053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:15:55.447053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:15:55.447053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:15:55.448020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:15:55.448020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:15:55.448020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:15:55.448020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:15:55.448020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:55.448020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:55.448020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:55.448020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:55.448020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:55.448020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:55.448020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:55.448020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:55.448020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:55.448020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:55.448020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:55.448020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:55.452492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:55.452492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:55.452492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:55.452492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:55.452492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:15:55.452492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:15:55.452492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:15:55.452492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:15:55.452492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:15:55.452492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:15:55.452492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:15:55.452492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:55.452492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 523/1042] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:15:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:15:55.454834 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ccbab2-7062-4f5b-a72d-60bbc4a1bc1d None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:55.505460 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:55.538964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 520/1043] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:15:55 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 04:15:55.544655 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e29078b-f41e-4cb1-a022-4b9160b7f25d None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:55.572820 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ccbab2-7062-4f5b-a72d-60bbc4a1bc1d None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:55.594571 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:55.614380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7327efd-4cce-4693-ad31-dd42af3c888d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:55.615944 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7327efd-4cce-4693-ad31-dd42af3c888d None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:55.616427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7327efd-4cce-4693-ad31-dd42af3c888d None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:55.616915 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7327efd-4cce-4693-ad31-dd42af3c888d None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:55.617776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7327efd-4cce-4693-ad31-dd42af3c888d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:55.618443 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7327efd-4cce-4693-ad31-dd42af3c888d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:55.620886 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7327efd-4cce-4693-ad31-dd42af3c888d None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:55.630950 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 524/1044] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:15:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:55.636770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7327efd-4cce-4693-ad31-dd42af3c888d None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:55.656386 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e29078b-f41e-4cb1-a022-4b9160b7f25d None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:55.695139 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a891674-0c27-462d-bc91-3332408f5977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:55.695794 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a891674-0c27-462d-bc91-3332408f5977 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:55.696121 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a891674-0c27-462d-bc91-3332408f5977 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:55.696475 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a891674-0c27-462d-bc91-3332408f5977 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:55.707369 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a891674-0c27-462d-bc91-3332408f5977 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:55.708984 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a891674-0c27-462d-bc91-3332408f5977 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:55.724498 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-7a891674-0c27-462d-bc91-3332408f5977 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:55.724498 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:55.724498 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:55.724498 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:55.724498 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:55.724498 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:55.724498 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:55.724498 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:55.724498 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:55.724498 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:55.724498 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:55.724498 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:55.724498 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:55.724498 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:55.724498 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:55.724498 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:55.725442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:55.725442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:55.725442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:55.725442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:55.725442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:55.725442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:55.725442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:55.725442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:55.725442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:55.725442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:55.725442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:55.725442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:55.725442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:55.725442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:55.725442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:55.725442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:55.725442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:55.726918 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:55.726918 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:55.726918 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:55.726918 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 521/1045] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:15:55.731144 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a891674-0c27-462d-bc91-3332408f5977 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:55.743725 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7327efd-4cce-4693-ad31-dd42af3c888d None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:55.756276 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:55.780708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26949a-5456-4dbd-a4ac-dcfdcf9b9a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:55.781346 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26949a-5456-4dbd-a4ac-dcfdcf9b9a51 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:55.781628 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26949a-5456-4dbd-a4ac-dcfdcf9b9a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:55.781902 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26949a-5456-4dbd-a4ac-dcfdcf9b9a51 None admin] PATH_INFO: `/v3/users/01c4881f511c4bd1940af46843c6f558` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:55.796112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b26949a-5456-4dbd-a4ac-dcfdcf9b9a51 None admin] RBAC: Authorizing `identity:get_user(user_id=01c4881f511c4bd1940af46843c6f558)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:55.797536 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b26949a-5456-4dbd-a4ac-dcfdcf9b9a51 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:55.799354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 525/1046] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:15:55 2022] GET /identity/v3/users/01c4881f511c4bd1940af46843c6f558 => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:55.805321 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26949a-5456-4dbd-a4ac-dcfdcf9b9a51 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:55.834176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a891674-0c27-462d-bc91-3332408f5977 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:55.850379 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:55.873146 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f99af-3975-4c93-a757-79c943f035bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:55.873807 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f99af-3975-4c93-a757-79c943f035bf None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:55.874094 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f99af-3975-4c93-a757-79c943f035bf None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:55.874352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f99af-3975-4c93-a757-79c943f035bf None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:55.875088 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f99af-3975-4c93-a757-79c943f035bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:55.875391 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f99af-3975-4c93-a757-79c943f035bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:55.876916 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f99af-3975-4c93-a757-79c943f035bf None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:55.893574 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 522/1047] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:55.894616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f99af-3975-4c93-a757-79c943f035bf None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:55.913642 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26949a-5456-4dbd-a4ac-dcfdcf9b9a51 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:55.950741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20458feb-d027-4e5a-9126-118bb87fbdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:55.951355 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20458feb-d027-4e5a-9126-118bb87fbdf4 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:55.951633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20458feb-d027-4e5a-9126-118bb87fbdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:55.951904 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20458feb-d027-4e5a-9126-118bb87fbdf4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:55.979330 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:55.997629 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f99af-3975-4c93-a757-79c943f035bf None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:56.030081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20458feb-d027-4e5a-9126-118bb87fbdf4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:56.031534 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20458feb-d027-4e5a-9126-118bb87fbdf4 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:56.032487 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d5623-d858-4206-bd3a-02f57a56a055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:56.033110 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d5623-d858-4206-bd3a-02f57a56a055 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:56.033386 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d5623-d858-4206-bd3a-02f57a56a055 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:56.033672 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d5623-d858-4206-bd3a-02f57a56a055 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:56.041463 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d5623-d858-4206-bd3a-02f57a56a055 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:56.042905 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d5623-d858-4206-bd3a-02f57a56a055 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:56.053778 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-723d5623-d858-4206-bd3a-02f57a56a055 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:56.053778 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:56.053778 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:56.053778 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:56.053778 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:56.053778 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:56.053778 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:56.053778 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:56.053778 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:56.053778 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:56.053778 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:56.053778 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:56.053778 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:15:56.053778 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:15:56.053778 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:15:56.054683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:15:56.054683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:56.054683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:56.054683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:56.054683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:56.054683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:56.054683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:56.054683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:56.054683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:56.054683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:56.054683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:56.054683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:56.054683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:56.054683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:56.054683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:56.054683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:56.054683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:56.055568 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:15:56.055568 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:15:56.055568 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:15:56.055568 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:15:56.055568 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:15:56.055568 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:15:56.055568 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:15:56.055568 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:56.055568 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 523/1048] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:15:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:56.060805 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d5623-d858-4206-bd3a-02f57a56a055 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:56.078088 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:56.141318 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 526/1049] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:15:55 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 04:15:56.146826 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20458feb-d027-4e5a-9126-118bb87fbdf4 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:56.168955 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d5623-d858-4206-bd3a-02f57a56a055 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:56.202296 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423137cc-a5c0-42ba-a5d4-a50700ed63f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:56.202934 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423137cc-a5c0-42ba-a5d4-a50700ed63f2 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:56.203209 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423137cc-a5c0-42ba-a5d4-a50700ed63f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:56.203481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423137cc-a5c0-42ba-a5d4-a50700ed63f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:56.204194 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423137cc-a5c0-42ba-a5d4-a50700ed63f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:56.204495 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423137cc-a5c0-42ba-a5d4-a50700ed63f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:56.205936 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423137cc-a5c0-42ba-a5d4-a50700ed63f2 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:56.219815 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 524/1050] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:15:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:56.226116 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423137cc-a5c0-42ba-a5d4-a50700ed63f2 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:56.228843 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:56.250244 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20458feb-d027-4e5a-9126-118bb87fbdf4 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:56.285427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b601b1-4d56-4e63-8b98-967e9ed91af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:56.286456 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b601b1-4d56-4e63-8b98-967e9ed91af2 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:56.286736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b601b1-4d56-4e63-8b98-967e9ed91af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:56.287010 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b601b1-4d56-4e63-8b98-967e9ed91af2 None admin] PATH_INFO: `/v3/projects/5fbe3eccfc48472693f4466cbfd55615` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:56.288145 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b601b1-4d56-4e63-8b98-967e9ed91af2 None admin] RBAC: Authorizing `identity:get_project(project_id=5fbe3eccfc48472693f4466cbfd55615)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:56.289575 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b601b1-4d56-4e63-8b98-967e9ed91af2 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:56.291209 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 527/1051] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:15:56 2022] GET /identity/v3/projects/5fbe3eccfc48472693f4466cbfd55615 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:56.297457 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b601b1-4d56-4e63-8b98-967e9ed91af2 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:56.309148 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:56.332952 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423137cc-a5c0-42ba-a5d4-a50700ed63f2 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:56.369742 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8934995-d6cc-4c0d-9426-b727ee552acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:56.370342 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8934995-d6cc-4c0d-9426-b727ee552acc None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:56.370596 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8934995-d6cc-4c0d-9426-b727ee552acc None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:56.370877 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8934995-d6cc-4c0d-9426-b727ee552acc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:56.371598 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8934995-d6cc-4c0d-9426-b727ee552acc None admin] RBAC: Adding query filter params (role.id=3631d69e831a474b807aeca180b214dc, scope.project.id=9f4373dfb5f9425eb73669e24d18a08f, user.id=cd7b11f479a54adc8d9b8641859b07a1) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:56.371886 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8934995-d6cc-4c0d-9426-b727ee552acc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:56.373341 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8934995-d6cc-4c0d-9426-b727ee552acc None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:56.387945 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 525/1052] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:15:56 2022] GET /identity/v3/role_assignments?user.id=cd7b11f479a54adc8d9b8641859b07a1&scope.project.id=9f4373dfb5f9425eb73669e24d18a08f&role.id=3631d69e831a474b807aeca180b214dc => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:56.415700 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b601b1-4d56-4e63-8b98-967e9ed91af2 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:56.460919 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e2f99-549c-4897-8eba-8fda061a69f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:56.460919 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e2f99-549c-4897-8eba-8fda061a69f1 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:56.460919 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e2f99-549c-4897-8eba-8fda061a69f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:56.460919 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e2f99-549c-4897-8eba-8fda061a69f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:56.461310 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19e2f99-549c-4897-8eba-8fda061a69f1 None admin] RBAC: Adding query filter params (role.id=52b31061b72c45ccb76dd26bf31f8d3f, scope.project.id=5fbe3eccfc48472693f4466cbfd55615, user.id=01c4881f511c4bd1940af46843c6f558) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:56.461310 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19e2f99-549c-4897-8eba-8fda061a69f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:56.462453 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19e2f99-549c-4897-8eba-8fda061a69f1 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:56.475969 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 528/1053] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:15:56 2022] GET /identity/v3/role_assignments?user.id=01c4881f511c4bd1940af46843c6f558&scope.project.id=5fbe3eccfc48472693f4466cbfd55615&role.id=52b31061b72c45ccb76dd26bf31f8d3f => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:56.505409 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b50c4-be50-4893-b843-f4ef3e852bbf None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:56.505969 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b50c4-be50-4893-b843-f4ef3e852bbf None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:56.505969 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b50c4-be50-4893-b843-f4ef3e852bbf None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:56.507583 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 526/1054] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:56.513475 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e05cc-927b-419c-96bf-71207018f0ce None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:56.513827 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e05cc-927b-419c-96bf-71207018f0ce None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:56.514047 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e05cc-927b-419c-96bf-71207018f0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:56.534650 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-de0e05cc-927b-419c-96bf-71207018f0ce None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:56.536013 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-de0e05cc-927b-419c-96bf-71207018f0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:56.612356 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 529/1055] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:15:56.877961 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b50c4-be50-4893-b843-f4ef3e852bbf None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:56.950103 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:56.988568 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b50c4-be50-4893-b843-f4ef3e852bbf None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:57.024404 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a80d3d-785b-4334-8e2b-3ad302e65c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:57.025040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a80d3d-785b-4334-8e2b-3ad302e65c40 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:57.025312 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a80d3d-785b-4334-8e2b-3ad302e65c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:57.025577 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a80d3d-785b-4334-8e2b-3ad302e65c40 None admin] PATH_INFO: `/v3/services/image` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:57.026157 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a80d3d-785b-4334-8e2b-3ad302e65c40 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:57.027518 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a80d3d-785b-4334-8e2b-3ad302e65c40 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:57.036602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-e7a80d3d-785b-4334-8e2b-3ad302e65c40 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 04:15:57.036602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:57.036602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 04:15:57.036602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 04:15:57.036602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 04:15:57.036602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 04:15:57.036602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 04:15:57.036602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:57.036602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 04:15:57.036602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:57.036602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:15:57.036602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:57.036602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:57.036602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:57.036602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:57.036602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:57.036602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:57.037546 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:57.037546 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:57.037546 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:57.037546 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:57.037546 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:57.037546 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:57.037546 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 04:15:57.037546 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 04:15:57.037546 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 04:15:57.037546 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 04:15:57.037546 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:57.037546 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:57.037546 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:57.037546 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:57.037546 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:57.037546 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:57.040948 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:57.040948 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:57.040948 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:57.040948 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:57.040948 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:57.040948 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:57.040948 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:57.040948 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:57.040948 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:57.040948 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:57.040948 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 04:15:57.040948 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:15:57.040948 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 04:15:57.040948 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:57.040948 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 527/1056] 10.222.0.18 () {56 vars in 1105 bytes} [Tue Aug 23 04:15:56 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:57.043706 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e05cc-927b-419c-96bf-71207018f0ce None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:57.046107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:57.148515 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e05cc-927b-419c-96bf-71207018f0ce None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:57.181798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02459ddf-5f0d-4c0d-9a19-b50f482def4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:57.182402 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02459ddf-5f0d-4c0d-9a19-b50f482def4c None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:57.182671 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02459ddf-5f0d-4c0d-9a19-b50f482def4c None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:57.182929 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02459ddf-5f0d-4c0d-9a19-b50f482def4c None admin] PATH_INFO: `/v3/services` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:57.183604 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02459ddf-5f0d-4c0d-9a19-b50f482def4c None admin] RBAC: Adding query filter params (name=image) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:57.183900 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02459ddf-5f0d-4c0d-9a19-b50f482def4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:57.185282 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02459ddf-5f0d-4c0d-9a19-b50f482def4c None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:57.194601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 530/1057] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:57.195825 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a80d3d-785b-4334-8e2b-3ad302e65c40 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:57.298161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a80d3d-785b-4334-8e2b-3ad302e65c40 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:57.325602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:57.332803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69328252-7997-480f-9fef-3119c146384e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:57.333751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69328252-7997-480f-9fef-3119c146384e None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:57.334029 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69328252-7997-480f-9fef-3119c146384e None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:57.334294 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69328252-7997-480f-9fef-3119c146384e None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:57.334922 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69328252-7997-480f-9fef-3119c146384e None admin] RBAC: Adding query filter params (type=image) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:57.335219 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69328252-7997-480f-9fef-3119c146384e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:57.336620 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69328252-7997-480f-9fef-3119c146384e None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:57.342575 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 528/1058] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:57 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:57.348260 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02459ddf-5f0d-4c0d-9a19-b50f482def4c None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:57.412793 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:57.449152 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02459ddf-5f0d-4c0d-9a19-b50f482def4c None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:57.490508 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3305c5d0-f7f9-4add-abdf-702bf5db6dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:57.491128 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305c5d0-f7f9-4add-abdf-702bf5db6dfe None admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:57.491407 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305c5d0-f7f9-4add-abdf-702bf5db6dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:57.491677 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305c5d0-f7f9-4add-abdf-702bf5db6dfe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:57.492373 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3305c5d0-f7f9-4add-abdf-702bf5db6dfe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:57.493743 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3305c5d0-f7f9-4add-abdf-702bf5db6dfe None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:57.523953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 531/1059] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:15:57 2022] POST /identity/v3/endpoints => generated 327 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:15:58.280450 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06f7fd-c1b0-4694-9890-432146fea963 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:58.280981 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06f7fd-c1b0-4694-9890-432146fea963 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:58.280981 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06f7fd-c1b0-4694-9890-432146fea963 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:58.282364 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 529/1060] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:58.287843 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d66fe-dca0-4e00-b1b6-2ec629d950f5 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:58.288276 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d66fe-dca0-4e00-b1b6-2ec629d950f5 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:58.288601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d66fe-dca0-4e00-b1b6-2ec629d950f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:58.307300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-db2d66fe-dca0-4e00-b1b6-2ec629d950f5 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:58.308515 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-db2d66fe-dca0-4e00-b1b6-2ec629d950f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:58.383586 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 532/1061] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:15:58.413280 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9008bf0e-afd0-4472-bf5b-59b14d9c557f None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:58.413568 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9008bf0e-afd0-4472-bf5b-59b14d9c557f None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:58.413845 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9008bf0e-afd0-4472-bf5b-59b14d9c557f None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:58.415358 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 530/1062] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:58.420494 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d9d2d-b77c-41bb-a537-f87419fbbd53 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:58.420762 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d9d2d-b77c-41bb-a537-f87419fbbd53 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:58.421067 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d9d2d-b77c-41bb-a537-f87419fbbd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:58.439540 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-ff6d9d2d-b77c-41bb-a537-f87419fbbd53 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:58.440759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-ff6d9d2d-b77c-41bb-a537-f87419fbbd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:58.493208 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:58.518573 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 533/1063] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:15:58.652136 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9008bf0e-afd0-4472-bf5b-59b14d9c557f None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:58.761262 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9008bf0e-afd0-4472-bf5b-59b14d9c557f None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:58.783474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d9d2d-b77c-41bb-a537-f87419fbbd53 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:58.798330 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f2e3e-6086-413a-9d5a-32bbe58117af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:58.799280 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f2e3e-6086-413a-9d5a-32bbe58117af None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:58.799578 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f2e3e-6086-413a-9d5a-32bbe58117af None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:58.799864 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f2e3e-6086-413a-9d5a-32bbe58117af None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:58.811060 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f2e3e-6086-413a-9d5a-32bbe58117af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:15:58.815668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-551f2e3e-6086-413a-9d5a-32bbe58117af None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 04:15:58.815668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:58.815668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:58.815668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:58.815668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:58.815668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:58.815668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:58.815668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:58.815668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:58.815668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:58.815668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:58.815668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:58.815668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:15:58.815668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:15:58.815668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:15:58.815668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:15:58.816589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:15:58.816589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:15:58.816589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:15:58.816589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:15:58.816589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:58.816589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:58.816589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:58.816589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:58.816589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:58.816589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:58.816589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:58.816589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:58.816589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:58.816589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:58.816589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:58.816589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:58.820647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:58.820647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:58.820647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:58.820647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:58.820647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:15:58.820647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:15:58.820647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:15:58.820647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:15:58.820647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:15:58.820647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:15:58.820647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 04:15:58.820647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:58.820647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 531/1064] 10.222.0.18 () {56 vars in 1120 bytes} [Tue Aug 23 04:15:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:58.822929 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f2e3e-6086-413a-9d5a-32bbe58117af None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:58.882735 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:58.900223 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d9d2d-b77c-41bb-a537-f87419fbbd53 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:58.930971 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f2e3e-6086-413a-9d5a-32bbe58117af None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:58.940567 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5733d5-07af-4c8d-b943-8a12e979d13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:58.941315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5733d5-07af-4c8d-b943-8a12e979d13a None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:58.941606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5733d5-07af-4c8d-b943-8a12e979d13a None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:58.942289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5733d5-07af-4c8d-b943-8a12e979d13a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:58.953212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5733d5-07af-4c8d-b943-8a12e979d13a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:15:58.958421 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-5a5733d5-07af-4c8d-b943-8a12e979d13a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:15:58.958421 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:58.958421 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:58.958421 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:58.958421 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:58.958421 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:58.958421 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:58.958421 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:58.958421 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:58.958421 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:58.958421 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:58.958421 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:58.958421 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:15:58.958421 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:15:58.958421 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:15:58.958421 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:15:58.959383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:15:58.959383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:15:58.959383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:15:58.959383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:15:58.959383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:58.959383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:58.959383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:58.959383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:58.959383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:58.959383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:58.959383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:58.959383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:58.959383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:58.959383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:58.959383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:58.959383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:58.960952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:58.960952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:58.960952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:58.960952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:58.960952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:15:58.960952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:15:58.960952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:15:58.960952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:15:58.960952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:15:58.960952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:15:58.960952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:15:58.960952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:58.960952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 534/1065] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:15:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:58.965756 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5733d5-07af-4c8d-b943-8a12e979d13a None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:58.971290 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075396b-54da-4c84-9abc-a9a7f8ff6065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:58.972144 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075396b-54da-4c84-9abc-a9a7f8ff6065 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:58.972599 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075396b-54da-4c84-9abc-a9a7f8ff6065 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:58.973042 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075396b-54da-4c84-9abc-a9a7f8ff6065 None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:58.973953 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075396b-54da-4c84-9abc-a9a7f8ff6065 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:58.974435 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075396b-54da-4c84-9abc-a9a7f8ff6065 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:58.976029 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075396b-54da-4c84-9abc-a9a7f8ff6065 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:58.985664 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 532/1066] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Aug 23 04:15:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:58.992543 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075396b-54da-4c84-9abc-a9a7f8ff6065 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:59.014766 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:59.087284 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5733d5-07af-4c8d-b943-8a12e979d13a None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:59.105091 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075396b-54da-4c84-9abc-a9a7f8ff6065 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:59.122925 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:59.127595 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7200397e-33e4-4a67-b943-a506158b8a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:59.128039 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7200397e-33e4-4a67-b943-a506158b8a5e None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:59.128377 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7200397e-33e4-4a67-b943-a506158b8a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:59.128651 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7200397e-33e4-4a67-b943-a506158b8a5e None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:59.129297 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7200397e-33e4-4a67-b943-a506158b8a5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:59.129669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7200397e-33e4-4a67-b943-a506158b8a5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:59.131063 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7200397e-33e4-4a67-b943-a506158b8a5e None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:59.140145 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 535/1067] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:15:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:59.144533 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea992b8-e711-4ea2-b298-f72ff50451b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:59.145652 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea992b8-e711-4ea2-b298-f72ff50451b7 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:59.145827 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7200397e-33e4-4a67-b943-a506158b8a5e None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:59.145960 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea992b8-e711-4ea2-b298-f72ff50451b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:59.146227 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea992b8-e711-4ea2-b298-f72ff50451b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:59.157287 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea992b8-e711-4ea2-b298-f72ff50451b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:59.158742 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea992b8-e711-4ea2-b298-f72ff50451b7 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:59.173806 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-1ea992b8-e711-4ea2-b298-f72ff50451b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:59.173806 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:59.173806 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:59.173806 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:59.173806 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:59.173806 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:59.173806 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:59.173806 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:59.173806 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:59.173806 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:59.173806 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:59.173806 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:59.173806 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:59.173806 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:59.173806 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:59.173806 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:59.174958 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:59.174958 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:59.174958 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:59.174958 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:59.174958 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:59.174958 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:59.174958 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:59.174958 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:59.174958 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:59.174958 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:59.174958 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:59.174958 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:59.174958 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:59.174958 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:59.174958 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:59.174958 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:59.174958 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:59.176070 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:59.176070 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:59.176070 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:59.181457 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 533/1068] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:59.182166 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea992b8-e711-4ea2-b298-f72ff50451b7 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:59.254381 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:59.258361 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7200397e-33e4-4a67-b943-a506158b8a5e None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:59.287505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea992b8-e711-4ea2-b298-f72ff50451b7 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:59.296458 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873f510e-1059-4890-be98-6e7ac65756cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:59.297361 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f510e-1059-4890-be98-6e7ac65756cc None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:59.297473 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f510e-1059-4890-be98-6e7ac65756cc None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:59.297749 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f510e-1059-4890-be98-6e7ac65756cc None admin] PATH_INFO: `/v3/users/01c4881f511c4bd1940af46843c6f558` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:59.298988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873f510e-1059-4890-be98-6e7ac65756cc None admin] RBAC: Authorizing `identity:get_user(user_id=01c4881f511c4bd1940af46843c6f558)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:59.300454 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873f510e-1059-4890-be98-6e7ac65756cc None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:59.307313 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 536/1069] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:15:59 2022] GET /identity/v3/users/01c4881f511c4bd1940af46843c6f558 => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:59.307967 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873f510e-1059-4890-be98-6e7ac65756cc None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:59.322016 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e2ca1-3a15-4a64-9925-2338a3c260e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:59.322656 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e2ca1-3a15-4a64-9925-2338a3c260e6 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:59.322935 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e2ca1-3a15-4a64-9925-2338a3c260e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:59.323199 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e2ca1-3a15-4a64-9925-2338a3c260e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:59.323827 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e2ca1-3a15-4a64-9925-2338a3c260e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:59.324156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e2ca1-3a15-4a64-9925-2338a3c260e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:59.325526 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e2ca1-3a15-4a64-9925-2338a3c260e6 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:59.337343 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 534/1070] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:59.344034 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e2ca1-3a15-4a64-9925-2338a3c260e6 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:59.379461 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:59.412169 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873f510e-1059-4890-be98-6e7ac65756cc None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:59.447437 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537da9b6-32d1-40c4-9f7c-f0ba7a73040e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:59.448019 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537da9b6-32d1-40c4-9f7c-f0ba7a73040e None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:59.448290 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537da9b6-32d1-40c4-9f7c-f0ba7a73040e None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:59.448556 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537da9b6-32d1-40c4-9f7c-f0ba7a73040e None admin] PATH_INFO: `/v3/projects/5fbe3eccfc48472693f4466cbfd55615` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:59.449667 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537da9b6-32d1-40c4-9f7c-f0ba7a73040e None admin] RBAC: Authorizing `identity:get_project(project_id=5fbe3eccfc48472693f4466cbfd55615)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:59.451092 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537da9b6-32d1-40c4-9f7c-f0ba7a73040e None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:59.453037 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 537/1071] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:15:59 2022] GET /identity/v3/projects/5fbe3eccfc48472693f4466cbfd55615 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:15:59.454179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ed6cd-0e85-4df0-892a-e29782b81365 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:59.454460 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ed6cd-0e85-4df0-892a-e29782b81365 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:59.454732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ed6cd-0e85-4df0-892a-e29782b81365 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:59.456071 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 538/1072] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:15:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:15:59.460959 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ed6cd-0e85-4df0-892a-e29782b81365 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:59.467554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e2ca1-3a15-4a64-9925-2338a3c260e6 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:59.497381 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:59.500891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c26fd4c-48d5-4423-8329-8ee83dab11c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:59.501528 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c26fd4c-48d5-4423-8329-8ee83dab11c4 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:59.501807 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c26fd4c-48d5-4423-8329-8ee83dab11c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:59.502067 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c26fd4c-48d5-4423-8329-8ee83dab11c4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:59.509266 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c26fd4c-48d5-4423-8329-8ee83dab11c4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:59.510719 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c26fd4c-48d5-4423-8329-8ee83dab11c4 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:59.522319 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-1c26fd4c-48d5-4423-8329-8ee83dab11c4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 04:15:59.522319 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:59.522319 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:59.522319 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:59.522319 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:59.522319 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:59.522319 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:59.522319 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:59.522319 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:59.522319 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:59.522319 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:59.522319 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:59.522319 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:15:59.522319 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:15:59.522319 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:15:59.522319 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:15:59.523250 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:59.523250 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:59.523250 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:15:59.523250 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:59.523250 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:15:59.523250 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:15:59.523250 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:59.523250 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:59.523250 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:59.523250 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:59.523250 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:59.523250 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:59.523250 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:59.523250 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:59.523250 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:59.523250 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:59.523250 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:59.523250 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:59.524231 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:59.524231 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:59.524231 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:15:59.524231 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:15:59.524231 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:15:59.524231 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:15:59.524231 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:15:59.524231 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:15:59.524231 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 04:15:59.524231 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:15:59.524231 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 535/1073] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Aug 23 04:15:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:59.524816 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b5d8d-e7ba-4014-a4a1-bdd37943bacf None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:59.524998 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b5d8d-e7ba-4014-a4a1-bdd37943bacf None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:59.525261 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b5d8d-e7ba-4014-a4a1-bdd37943bacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:59.542805 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-ad4b5d8d-e7ba-4014-a4a1-bdd37943bacf None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:15:59.544030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-ad4b5d8d-e7ba-4014-a4a1-bdd37943bacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:15:59.569173 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ed6cd-0e85-4df0-892a-e29782b81365 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:59.603306 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122966f-421c-4222-8931-570f92d80b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:59.603979 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122966f-421c-4222-8931-570f92d80b26 None admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:59.604273 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122966f-421c-4222-8931-570f92d80b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:59.604547 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122966f-421c-4222-8931-570f92d80b26 None admin] PATH_INFO: `/v3/projects/5fbe3eccfc48472693f4466cbfd55615/users/01c4881f511c4bd1940af46843c6f558/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:59.606523 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9122966f-421c-4222-8931-570f92d80b26 None admin] RBAC: Authorizing `identity:create_grant(project_id=5fbe3eccfc48472693f4466cbfd55615, user_id=01c4881f511c4bd1940af46843c6f558, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:59.607986 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9122966f-421c-4222-8931-570f92d80b26 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:59.610041 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 536/1074] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:15:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:15:59.610853 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b5d8d-e7ba-4014-a4a1-bdd37943bacf None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:59.627950 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 539/1075] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Aug 23 04:15:59 2022] PUT /identity/v3/projects/5fbe3eccfc48472693f4466cbfd55615/users/01c4881f511c4bd1940af46843c6f558/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:15:59.638437 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:59.733487 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b5d8d-e7ba-4014-a4a1-bdd37943bacf None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:59.772851 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e00c7-44d3-4c92-b220-1b87bdc4e85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:59.773086 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e00c7-44d3-4c92-b220-1b87bdc4e85f None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:59.773304 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e00c7-44d3-4c92-b220-1b87bdc4e85f None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:59.773489 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e00c7-44d3-4c92-b220-1b87bdc4e85f None admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:59.774248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e00c7-44d3-4c92-b220-1b87bdc4e85f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:15:59.774546 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e00c7-44d3-4c92-b220-1b87bdc4e85f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:59.776181 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e00c7-44d3-4c92-b220-1b87bdc4e85f None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:59.795335 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 537/1076] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Aug 23 04:15:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:15:59.800982 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122966f-421c-4222-8931-570f92d80b26 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:59.882031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e00c7-44d3-4c92-b220-1b87bdc4e85f None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:59.902682 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:59.909651 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122966f-421c-4222-8931-570f92d80b26 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:15:59.945097 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefd9def-08f0-44f5-8698-3d9a2a187d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:15:59.945744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefd9def-08f0-44f5-8698-3d9a2a187d44 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:15:59.946033 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefd9def-08f0-44f5-8698-3d9a2a187d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:15:59.946306 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefd9def-08f0-44f5-8698-3d9a2a187d44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:15:59.957367 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefd9def-08f0-44f5-8698-3d9a2a187d44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:15:59.959091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefd9def-08f0-44f5-8698-3d9a2a187d44 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:15:59.971404 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:15:59.977977 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-aefd9def-08f0-44f5-8698-3d9a2a187d44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:59.977977 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:15:59.977977 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:15:59.977977 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:15:59.977977 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:15:59.977977 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:15:59.977977 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:15:59.977977 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:15:59.977977 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:15:59.977977 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:15:59.977977 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:15:59.977977 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:15:59.977977 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:15:59.977977 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:15:59.977977 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:15:59.977977 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:15:59.978954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:15:59.978954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:15:59.978954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:15:59.978954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:15:59.978954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:15:59.978954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:15:59.978954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:15:59.978954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:15:59.978954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:15:59.978954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:15:59.978954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:15:59.978954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:15:59.978954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:15:59.978954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:15:59.978954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:15:59.978954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:15:59.978954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:15:59.979826 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:15:59.979826 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:15:59.979826 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:15:59.979826 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 540/1077] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:15:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:15:59.985019 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefd9def-08f0-44f5-8698-3d9a2a187d44 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:15:59.999083 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e00c7-44d3-4c92-b220-1b87bdc4e85f None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:00.034716 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02adcd6-a20a-430f-99eb-dc213a0e8f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:00.035383 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02adcd6-a20a-430f-99eb-dc213a0e8f00 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:00.035863 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02adcd6-a20a-430f-99eb-dc213a0e8f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:00.036287 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02adcd6-a20a-430f-99eb-dc213a0e8f00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:00.092683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefd9def-08f0-44f5-8698-3d9a2a187d44 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:00.109853 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02adcd6-a20a-430f-99eb-dc213a0e8f00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:00.111329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02adcd6-a20a-430f-99eb-dc213a0e8f00 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:00.128593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803e52b-19fb-44a4-8e8b-88a03173f907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:00.129214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803e52b-19fb-44a4-8e8b-88a03173f907 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:00.129572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803e52b-19fb-44a4-8e8b-88a03173f907 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:00.129841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803e52b-19fb-44a4-8e8b-88a03173f907 None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:00.130487 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5803e52b-19fb-44a4-8e8b-88a03173f907 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:16:00.130799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5803e52b-19fb-44a4-8e8b-88a03173f907 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:00.132197 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5803e52b-19fb-44a4-8e8b-88a03173f907 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:00.144918 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 541/1078] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:16:00.150626 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803e52b-19fb-44a4-8e8b-88a03173f907 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:00.161655 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:00.190547 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:00.219973 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 538/1079] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:15:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 04:16:00.222829 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02adcd6-a20a-430f-99eb-dc213a0e8f00 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:00.257311 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803e52b-19fb-44a4-8e8b-88a03173f907 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:00.292169 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721533a9-eced-4692-a9c7-6aff32b6e76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:00.292812 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721533a9-eced-4692-a9c7-6aff32b6e76a None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:00.293087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721533a9-eced-4692-a9c7-6aff32b6e76a None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:00.293354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721533a9-eced-4692-a9c7-6aff32b6e76a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:00.300176 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721533a9-eced-4692-a9c7-6aff32b6e76a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:00.301658 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721533a9-eced-4692-a9c7-6aff32b6e76a None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:00.312933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-721533a9-eced-4692-a9c7-6aff32b6e76a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:16:00.312933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:16:00.312933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:16:00.312933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:16:00.312933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:16:00.312933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:16:00.312933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:16:00.312933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:16:00.312933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:16:00.312933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:16:00.312933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:16:00.312933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:16:00.312933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:16:00.312933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:16:00.312933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:16:00.313806 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:16:00.313806 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:16:00.313806 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:16:00.313806 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:16:00.313806 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:16:00.313806 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:16:00.313806 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:16:00.313806 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:16:00.313806 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:16:00.313806 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:16:00.313806 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:16:00.313806 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:16:00.313806 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:16:00.313806 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:16:00.313806 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:16:00.313806 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:16:00.313806 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:16:00.314722 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:16:00.314722 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:16:00.314722 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:16:00.314722 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:16:00.314722 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:16:00.314722 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:16:00.314722 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:16:00.314722 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:16:00.314722 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 542/1080] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:16:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:16:00.320330 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721533a9-eced-4692-a9c7-6aff32b6e76a None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:00.329213 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02adcd6-a20a-430f-99eb-dc213a0e8f00 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:00.365173 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f970f3a8-2126-4fdb-8967-7bfb21103a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:00.365818 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f970f3a8-2126-4fdb-8967-7bfb21103a7d None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:00.366155 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f970f3a8-2126-4fdb-8967-7bfb21103a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:00.366412 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f970f3a8-2126-4fdb-8967-7bfb21103a7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:00.376761 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f970f3a8-2126-4fdb-8967-7bfb21103a7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:00.378220 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f970f3a8-2126-4fdb-8967-7bfb21103a7d None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:00.390706 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:00.392458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-f970f3a8-2126-4fdb-8967-7bfb21103a7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:16:00.392458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:16:00.392458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:16:00.392458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:16:00.392458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:16:00.392458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:16:00.392458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:16:00.392458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:16:00.392458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:16:00.392458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:16:00.392458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:16:00.392458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:16:00.392458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:16:00.392458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:16:00.392458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:16:00.392458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:16:00.393377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:16:00.393377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:16:00.393377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:16:00.393377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:16:00.393377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:16:00.393377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:16:00.393377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:16:00.393377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:16:00.393377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:16:00.393377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:16:00.393377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:16:00.393377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:16:00.393377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:16:00.393377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:16:00.393377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:16:00.393377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:16:00.393377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:16:00.394279 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:16:00.394279 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:16:00.394279 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:16:00.394279 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 539/1081] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:16:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:16:00.399689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f970f3a8-2126-4fdb-8967-7bfb21103a7d None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:00.429935 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:00.433421 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721533a9-eced-4692-a9c7-6aff32b6e76a None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:00.471209 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7629bae-bde7-4fa4-b674-ce21fa5198e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:00.471817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7629bae-bde7-4fa4-b674-ce21fa5198e6 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:00.472126 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7629bae-bde7-4fa4-b674-ce21fa5198e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:00.472413 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7629bae-bde7-4fa4-b674-ce21fa5198e6 None admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:00.473408 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7629bae-bde7-4fa4-b674-ce21fa5198e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:16:00.473517 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7629bae-bde7-4fa4-b674-ce21fa5198e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:00.475014 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7629bae-bde7-4fa4-b674-ce21fa5198e6 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:00.487367 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 543/1082] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:16:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:16:00.496609 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7629bae-bde7-4fa4-b674-ce21fa5198e6 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:00.528845 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f970f3a8-2126-4fdb-8967-7bfb21103a7d None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:00.565843 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6019c6-a796-4504-a71f-c48587bf31bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:00.566455 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6019c6-a796-4504-a71f-c48587bf31bb None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:00.566733 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6019c6-a796-4504-a71f-c48587bf31bb None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:00.567142 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6019c6-a796-4504-a71f-c48587bf31bb None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:00.568066 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6019c6-a796-4504-a71f-c48587bf31bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:16:00.568373 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6019c6-a796-4504-a71f-c48587bf31bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:00.569746 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6019c6-a796-4504-a71f-c48587bf31bb None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:00.580968 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 540/1083] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:16:00.615301 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7629bae-bde7-4fa4-b674-ce21fa5198e6 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:00.647796 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:00.657375 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a233912-1f99-41bb-acc0-4488f3438482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:00.658368 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a233912-1f99-41bb-acc0-4488f3438482 None admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:00.658876 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a233912-1f99-41bb-acc0-4488f3438482 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:00.659349 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a233912-1f99-41bb-acc0-4488f3438482 None admin] PATH_INFO: `/v3/projects/9f4373dfb5f9425eb73669e24d18a08f/users/cd7b11f479a54adc8d9b8641859b07a1/roles/3631d69e831a474b807aeca180b214dc` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:00.662022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a233912-1f99-41bb-acc0-4488f3438482 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f4373dfb5f9425eb73669e24d18a08f, user_id=cd7b11f479a54adc8d9b8641859b07a1, role_id=3631d69e831a474b807aeca180b214dc)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:00.663729 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a233912-1f99-41bb-acc0-4488f3438482 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:00.682137 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 544/1084] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Aug 23 04:16:00 2022] PUT /identity/v3/projects/9f4373dfb5f9425eb73669e24d18a08f/users/cd7b11f479a54adc8d9b8641859b07a1/roles/3631d69e831a474b807aeca180b214dc => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:16:01.581868 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68449f-e0b2-4e30-bfa9-599b78358f50 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:01.582397 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68449f-e0b2-4e30-bfa9-599b78358f50 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:01.582397 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68449f-e0b2-4e30-bfa9-599b78358f50 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:01.583813 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 541/1085] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:16:01.589760 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113016eb-7f00-4749-9cb0-aaafd2fa5dcd None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:01.590146 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113016eb-7f00-4749-9cb0-aaafd2fa5dcd None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:01.590342 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113016eb-7f00-4749-9cb0-aaafd2fa5dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:01.609757 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-113016eb-7f00-4749-9cb0-aaafd2fa5dcd None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:16:01.610960 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-113016eb-7f00-4749-9cb0-aaafd2fa5dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:16:01.682441 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 545/1086] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:16:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:16:01.942210 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68449f-e0b2-4e30-bfa9-599b78358f50 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:02.038740 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:02.052741 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68449f-e0b2-4e30-bfa9-599b78358f50 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:02.088259 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4fde4-e2ee-42fc-a72d-a8721d4a1caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:02.088918 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4fde4-e2ee-42fc-a72d-a8721d4a1caf None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:02.089191 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4fde4-e2ee-42fc-a72d-a8721d4a1caf None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:02.089454 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4fde4-e2ee-42fc-a72d-a8721d4a1caf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:02.100953 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb4fde4-e2ee-42fc-a72d-a8721d4a1caf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:16:02.105610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-bbb4fde4-e2ee-42fc-a72d-a8721d4a1caf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:16:02.105610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:16:02.105610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:16:02.105610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:16:02.105610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:16:02.105610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:16:02.105610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:16:02.105610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:16:02.105610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:16:02.105610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:16:02.105610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:16:02.105610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:16:02.105610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:16:02.105610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:16:02.105610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:16:02.105610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:16:02.106668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:16:02.106668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:16:02.106668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:16:02.106668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:16:02.106668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:16:02.106668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:16:02.106668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:16:02.106668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:16:02.106668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:16:02.106668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:16:02.106668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:16:02.106668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:16:02.106668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:16:02.106668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:16:02.106668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:16:02.106668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:16:02.108923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:16:02.108923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:16:02.108923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:16:02.108923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:16:02.108923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:16:02.108923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:16:02.108923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:16:02.108923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:16:02.108923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:16:02.108923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:16:02.108923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:16:02.108923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:16:02.108923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 542/1087] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:16:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:16:02.112998 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113016eb-7f00-4749-9cb0-aaafd2fa5dcd None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:02.211009 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113016eb-7f00-4749-9cb0-aaafd2fa5dcd None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:02.230217 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:02.247704 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9126a3fc-30be-426c-89e5-0c1b54024b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:02.248474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126a3fc-30be-426c-89e5-0c1b54024b2e None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:02.248716 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126a3fc-30be-426c-89e5-0c1b54024b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:02.249011 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126a3fc-30be-426c-89e5-0c1b54024b2e None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:02.249668 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9126a3fc-30be-426c-89e5-0c1b54024b2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:16:02.249967 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9126a3fc-30be-426c-89e5-0c1b54024b2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:02.251323 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9126a3fc-30be-426c-89e5-0c1b54024b2e None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:02.260156 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 546/1088] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:16:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:02.266028 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4fde4-e2ee-42fc-a72d-a8721d4a1caf None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:02.370067 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4fde4-e2ee-42fc-a72d-a8721d4a1caf None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:02.404795 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3ce9a-74db-40d7-852f-56b40f5d9647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:02.405420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3ce9a-74db-40d7-852f-56b40f5d9647 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:02.405691 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3ce9a-74db-40d7-852f-56b40f5d9647 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:02.405979 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3ce9a-74db-40d7-852f-56b40f5d9647 None admin] PATH_INFO: `/v3/users/01c4881f511c4bd1940af46843c6f558` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:02.407123 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3ce9a-74db-40d7-852f-56b40f5d9647 None admin] RBAC: Authorizing `identity:get_user(user_id=01c4881f511c4bd1940af46843c6f558)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:02.408592 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3ce9a-74db-40d7-852f-56b40f5d9647 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:02.410174 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 543/1089] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:16:02 2022] GET /identity/v3/users/01c4881f511c4bd1940af46843c6f558 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:02.416716 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9126a3fc-30be-426c-89e5-0c1b54024b2e None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:02.522019 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9126a3fc-30be-426c-89e5-0c1b54024b2e None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:02.556538 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf5fd6c-1a5d-4353-a0dc-16ee4e27b1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:02.557141 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf5fd6c-1a5d-4353-a0dc-16ee4e27b1a6 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:02.557401 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf5fd6c-1a5d-4353-a0dc-16ee4e27b1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:02.557855 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf5fd6c-1a5d-4353-a0dc-16ee4e27b1a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:02.566086 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8329c-845a-4ff8-b6ea-c878fe735683 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:02.566342 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8329c-845a-4ff8-b6ea-c878fe735683 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:02.566673 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8329c-845a-4ff8-b6ea-c878fe735683 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:02.568307 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 544/1090] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:16:02.574244 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe6de7-31e0-4241-9dcb-e3c3faaec347 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:02.574783 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe6de7-31e0-4241-9dcb-e3c3faaec347 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:02.575061 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe6de7-31e0-4241-9dcb-e3c3faaec347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:02.594573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-57fe6de7-31e0-4241-9dcb-e3c3faaec347 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:16:02.595766 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-57fe6de7-31e0-4241-9dcb-e3c3faaec347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:16:02.606955 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:02.610108 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:02.630133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf5fd6c-1a5d-4353-a0dc-16ee4e27b1a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:02.631584 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf5fd6c-1a5d-4353-a0dc-16ee4e27b1a6 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:02.665567 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 545/1091] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:16:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:16:02.731643 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 547/1092] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:16:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 04:16:02.737706 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe6de7-31e0-4241-9dcb-e3c3faaec347 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:02.836893 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe6de7-31e0-4241-9dcb-e3c3faaec347 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:02.873075 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfdb46-658c-4e12-8929-b7dd52d16e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:02.873706 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfdb46-658c-4e12-8929-b7dd52d16e4a None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:02.873985 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfdb46-658c-4e12-8929-b7dd52d16e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:02.874258 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfdb46-658c-4e12-8929-b7dd52d16e4a None admin] PATH_INFO: `/v3/projects/5fbe3eccfc48472693f4466cbfd55615` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:02.875387 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bfdb46-658c-4e12-8929-b7dd52d16e4a None admin] RBAC: Authorizing `identity:get_project(project_id=5fbe3eccfc48472693f4466cbfd55615)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:02.876868 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bfdb46-658c-4e12-8929-b7dd52d16e4a None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:02.878495 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 546/1093] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:16:02 2022] GET /identity/v3/projects/5fbe3eccfc48472693f4466cbfd55615 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:02.884811 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf5fd6c-1a5d-4353-a0dc-16ee4e27b1a6 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:02.935001 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfdb46-658c-4e12-8929-b7dd52d16e4a None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:02.952185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:02.966883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:02.987144 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf5fd6c-1a5d-4353-a0dc-16ee4e27b1a6 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:03.028964 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aafc1f6-1eb9-4af8-ac93-d8676f95b987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:03.029578 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aafc1f6-1eb9-4af8-ac93-d8676f95b987 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:03.029857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aafc1f6-1eb9-4af8-ac93-d8676f95b987 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:03.030128 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aafc1f6-1eb9-4af8-ac93-d8676f95b987 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:03.030865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aafc1f6-1eb9-4af8-ac93-d8676f95b987 None admin] RBAC: Adding query filter params (role.id=52b31061b72c45ccb76dd26bf31f8d3f, scope.project.id=5fbe3eccfc48472693f4466cbfd55615, user.id=01c4881f511c4bd1940af46843c6f558) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:16:03.031172 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aafc1f6-1eb9-4af8-ac93-d8676f95b987 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:03.032693 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aafc1f6-1eb9-4af8-ac93-d8676f95b987 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:03.038838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 548/1094] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:16:02 2022] GET /identity/v3/role_assignments?user.id=01c4881f511c4bd1940af46843c6f558&scope.project.id=5fbe3eccfc48472693f4466cbfd55615&role.id=52b31061b72c45ccb76dd26bf31f8d3f => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:03.047748 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfdb46-658c-4e12-8929-b7dd52d16e4a None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:03.087511 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98394d5-1bbb-4968-ad5a-8ba508262e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:03.088529 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98394d5-1bbb-4968-ad5a-8ba508262e11 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:03.088802 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98394d5-1bbb-4968-ad5a-8ba508262e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:03.089072 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98394d5-1bbb-4968-ad5a-8ba508262e11 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:03.102683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98394d5-1bbb-4968-ad5a-8ba508262e11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:16:03.107398 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-b98394d5-1bbb-4968-ad5a-8ba508262e11 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 04:16:03.107398 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:16:03.107398 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:16:03.107398 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:16:03.107398 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:16:03.107398 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:16:03.107398 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:16:03.107398 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:16:03.107398 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:16:03.107398 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:16:03.107398 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:16:03.107398 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:16:03.107398 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:16:03.107398 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:16:03.107398 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:16:03.107398 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:16:03.108411 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:16:03.108411 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:16:03.108411 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:16:03.108411 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:16:03.108411 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:16:03.108411 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:16:03.108411 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:16:03.108411 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:16:03.108411 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:16:03.108411 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:16:03.108411 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:16:03.108411 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:16:03.108411 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:16:03.108411 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:16:03.108411 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:16:03.108411 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:16:03.109236 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:16:03.109236 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:16:03.109236 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:16:03.109236 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:16:03.109236 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:16:03.109236 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:16:03.109236 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:16:03.109236 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:16:03.109236 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:16:03.109236 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:16:03.109236 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 04:16:03.109236 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:16:03.109236 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 547/1095] 10.222.0.18 () {56 vars in 1120 bytes} [Tue Aug 23 04:16:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:16:03.115365 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aafc1f6-1eb9-4af8-ac93-d8676f95b987 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:03.223767 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aafc1f6-1eb9-4af8-ac93-d8676f95b987 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:03.257926 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0e316-ba8e-4860-94ca-c5efe42d1f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:03.258543 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0e316-ba8e-4860-94ca-c5efe42d1f6f None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:03.258824 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0e316-ba8e-4860-94ca-c5efe42d1f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:03.259096 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0e316-ba8e-4860-94ca-c5efe42d1f6f None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:03.259764 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c0e316-ba8e-4860-94ca-c5efe42d1f6f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:16:03.260145 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c0e316-ba8e-4860-94ca-c5efe42d1f6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:03.261517 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c0e316-ba8e-4860-94ca-c5efe42d1f6f None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:03.269541 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 549/1096] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Aug 23 04:16:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:03.275833 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98394d5-1bbb-4968-ad5a-8ba508262e11 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:03.350864 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:03.377546 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98394d5-1bbb-4968-ad5a-8ba508262e11 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:03.413738 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850c510-e8d8-4404-8f7c-842f3b70012d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:03.414366 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850c510-e8d8-4404-8f7c-842f3b70012d None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:03.414649 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850c510-e8d8-4404-8f7c-842f3b70012d None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:03.414926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850c510-e8d8-4404-8f7c-842f3b70012d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:03.425849 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d850c510-e8d8-4404-8f7c-842f3b70012d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:03.427372 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d850c510-e8d8-4404-8f7c-842f3b70012d None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:03.441599 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-d850c510-e8d8-4404-8f7c-842f3b70012d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:16:03.441599 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:16:03.441599 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:16:03.441599 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:16:03.441599 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:16:03.441599 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:16:03.441599 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:16:03.441599 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:16:03.441599 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:16:03.441599 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:16:03.441599 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:16:03.441599 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:16:03.441599 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:16:03.441599 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:16:03.441599 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:16:03.441599 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:16:03.442664 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:16:03.442664 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:16:03.442664 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:16:03.442664 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:16:03.442664 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:16:03.442664 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:16:03.442664 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:16:03.442664 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:16:03.442664 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:16:03.442664 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:16:03.442664 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:16:03.442664 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:16:03.442664 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:16:03.442664 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:16:03.442664 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:16:03.442664 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:16:03.442664 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:16:03.443710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:16:03.443710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:16:03.443710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:16:03.447551 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 548/1097] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:16:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:16:03.449004 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0e316-ba8e-4860-94ca-c5efe42d1f6f None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:03.458783 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:03.553826 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0e316-ba8e-4860-94ca-c5efe42d1f6f None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:03.588876 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e855d-9bf0-4279-8517-eb102c5ca1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:03.589478 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e855d-9bf0-4279-8517-eb102c5ca1bb None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:03.589758 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e855d-9bf0-4279-8517-eb102c5ca1bb None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:03.590029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e855d-9bf0-4279-8517-eb102c5ca1bb None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:03.590674 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e855d-9bf0-4279-8517-eb102c5ca1bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:16:03.590978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e855d-9bf0-4279-8517-eb102c5ca1bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:03.592389 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e855d-9bf0-4279-8517-eb102c5ca1bb None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:03.605218 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 550/1098] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:16:03.611243 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850c510-e8d8-4404-8f7c-842f3b70012d None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:03.716189 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850c510-e8d8-4404-8f7c-842f3b70012d None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:03.742091 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:03.751602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad75eb8-861b-4169-81eb-37da09ea6964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:03.752281 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad75eb8-861b-4169-81eb-37da09ea6964 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:03.752632 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad75eb8-861b-4169-81eb-37da09ea6964 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:03.752903 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad75eb8-861b-4169-81eb-37da09ea6964 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:03.760932 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad75eb8-861b-4169-81eb-37da09ea6964 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:03.762417 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad75eb8-861b-4169-81eb-37da09ea6964 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:03.774138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-8ad75eb8-861b-4169-81eb-37da09ea6964 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 04:16:03.774138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:16:03.774138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:16:03.774138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:16:03.774138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:16:03.774138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:16:03.774138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:16:03.774138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:16:03.774138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:16:03.774138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:16:03.774138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:16:03.774138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:16:03.774138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:16:03.774138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:16:03.774138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:16:03.774138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:16:03.775171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:16:03.775171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:16:03.775171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:16:03.775171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:16:03.775171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:16:03.775171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:16:03.775171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:16:03.775171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:16:03.775171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:16:03.775171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:16:03.775171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:16:03.775171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:16:03.775171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:16:03.775171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:16:03.775171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:16:03.775171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:16:03.775171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:16:03.775171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:16:03.776163 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:16:03.776163 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:16:03.776163 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:16:03.776163 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:16:03.776163 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:16:03.776163 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:16:03.776163 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:16:03.776163 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:16:03.776163 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 04:16:03.776163 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:16:03.776163 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 549/1099] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Aug 23 04:16:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:16:03.781451 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e855d-9bf0-4279-8517-eb102c5ca1bb None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:03.849558 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:03.885179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e855d-9bf0-4279-8517-eb102c5ca1bb None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:03.922520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296a99c-c62f-480c-bbcb-0cf622cf47cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:03.923297 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296a99c-c62f-480c-bbcb-0cf622cf47cb None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:03.923696 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296a99c-c62f-480c-bbcb-0cf622cf47cb None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:03.924112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296a99c-c62f-480c-bbcb-0cf622cf47cb None admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:03.925360 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2296a99c-c62f-480c-bbcb-0cf622cf47cb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:16:03.925753 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2296a99c-c62f-480c-bbcb-0cf622cf47cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:03.928171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2296a99c-c62f-480c-bbcb-0cf622cf47cb None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:03.946164 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 551/1100] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Aug 23 04:16:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:03.952156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad75eb8-861b-4169-81eb-37da09ea6964 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:04.056336 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad75eb8-861b-4169-81eb-37da09ea6964 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:04.093726 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38b416-2b51-4957-a507-e262453c9889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:04.094333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38b416-2b51-4957-a507-e262453c9889 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:04.094611 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38b416-2b51-4957-a507-e262453c9889 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:04.094884 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38b416-2b51-4957-a507-e262453c9889 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:04.159887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:04.174296 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b38b416-2b51-4957-a507-e262453c9889 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:04.175771 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b38b416-2b51-4957-a507-e262453c9889 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:04.275982 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 550/1101] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:16:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 04:16:04.281884 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296a99c-c62f-480c-bbcb-0cf622cf47cb None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:04.385496 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296a99c-c62f-480c-bbcb-0cf622cf47cb None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:04.409419 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:04.421948 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a5f85-3356-4350-9a0f-9c7a76d7e956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:04.422507 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a5f85-3356-4350-9a0f-9c7a76d7e956 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:04.422780 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a5f85-3356-4350-9a0f-9c7a76d7e956 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:04.423046 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a5f85-3356-4350-9a0f-9c7a76d7e956 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:04.433141 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a5f85-3356-4350-9a0f-9c7a76d7e956 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:04.434575 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a5f85-3356-4350-9a0f-9c7a76d7e956 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:04.448819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-490a5f85-3356-4350-9a0f-9c7a76d7e956 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:16:04.448819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:16:04.448819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:16:04.448819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:16:04.448819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:16:04.448819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:16:04.448819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:16:04.448819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:16:04.448819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:16:04.448819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:16:04.448819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:16:04.448819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:16:04.448819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:16:04.448819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:16:04.448819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:16:04.448819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:16:04.449727 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:16:04.449727 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:16:04.449727 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:16:04.449727 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:16:04.449727 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:16:04.449727 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:16:04.449727 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:16:04.449727 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:16:04.449727 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:16:04.449727 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:16:04.449727 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:16:04.449727 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:16:04.449727 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:16:04.449727 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:16:04.449727 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:16:04.449727 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:16:04.449727 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:16:04.450610 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:16:04.450610 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:16:04.450610 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:16:04.450610 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 552/1102] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:16:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:16:04.455838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38b416-2b51-4957-a507-e262453c9889 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:04.545213 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:04.561097 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38b416-2b51-4957-a507-e262453c9889 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:04.595578 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06bc552-34f9-4838-9a50-b4c8571274a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:04.596213 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bc552-34f9-4838-9a50-b4c8571274a2 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:04.596489 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bc552-34f9-4838-9a50-b4c8571274a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:04.596755 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bc552-34f9-4838-9a50-b4c8571274a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:04.597390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06bc552-34f9-4838-9a50-b4c8571274a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:16:04.597686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06bc552-34f9-4838-9a50-b4c8571274a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:04.599288 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06bc552-34f9-4838-9a50-b4c8571274a2 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:04.610465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 551/1103] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:16:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:04.616313 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a5f85-3356-4350-9a0f-9c7a76d7e956 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:04.718207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a5f85-3356-4350-9a0f-9c7a76d7e956 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:04.752871 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e063a6-cc0f-4e2f-9137-8dc3776141b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:04.753497 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e063a6-cc0f-4e2f-9137-8dc3776141b7 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:04.753775 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e063a6-cc0f-4e2f-9137-8dc3776141b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:04.754043 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e063a6-cc0f-4e2f-9137-8dc3776141b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:04.761001 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e063a6-cc0f-4e2f-9137-8dc3776141b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:04.762433 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e063a6-cc0f-4e2f-9137-8dc3776141b7 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:04.774027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-79e063a6-cc0f-4e2f-9137-8dc3776141b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:16:04.774027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:16:04.774027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:16:04.774027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:16:04.774027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:16:04.774027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:16:04.774027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:16:04.774027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:16:04.774027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:16:04.774027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:16:04.774027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:16:04.774027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:16:04.774027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:16:04.774027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:16:04.774027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:16:04.775105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:16:04.775105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:16:04.775105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:16:04.775105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:16:04.775105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:16:04.775105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:16:04.775105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:16:04.775105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:16:04.775105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:16:04.775105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:16:04.775105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:16:04.775105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:16:04.775105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:16:04.775105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:16:04.775105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:16:04.775105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:16:04.775105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:16:04.776029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:16:04.776029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:16:04.776029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:16:04.776029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:16:04.776029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:16:04.776029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:16:04.776029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:16:04.776029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:16:04.776029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 553/1104] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:16:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:16:04.781148 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06bc552-34f9-4838-9a50-b4c8571274a2 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:04.884558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06bc552-34f9-4838-9a50-b4c8571274a2 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:04.919082 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f2cf2-2976-4fcf-8f6e-2df6d40c10e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:04.919687 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f2cf2-2976-4fcf-8f6e-2df6d40c10e8 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:04.919967 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f2cf2-2976-4fcf-8f6e-2df6d40c10e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:04.920248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f2cf2-2976-4fcf-8f6e-2df6d40c10e8 None admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:04.920958 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f2cf2-2976-4fcf-8f6e-2df6d40c10e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:16:04.921260 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f2cf2-2976-4fcf-8f6e-2df6d40c10e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:04.922699 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f2cf2-2976-4fcf-8f6e-2df6d40c10e8 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:04.934942 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 552/1105] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:16:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:04.941021 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e063a6-cc0f-4e2f-9137-8dc3776141b7 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:04.960800 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37327b38-4199-4fc4-b7f8-6b53c08ab1fd None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:04.961089 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37327b38-4199-4fc4-b7f8-6b53c08ab1fd None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:04.961365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37327b38-4199-4fc4-b7f8-6b53c08ab1fd None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:04.962760 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 553/1106] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:16:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:16:04.968103 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85068fed-30f7-4f2e-8a33-244d771545d2 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:04.968439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85068fed-30f7-4f2e-8a33-244d771545d2 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:04.968667 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85068fed-30f7-4f2e-8a33-244d771545d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:04.981124 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:04.982854 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:04.989787 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-85068fed-30f7-4f2e-8a33-244d771545d2 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:16:04.991008 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-85068fed-30f7-4f2e-8a33-244d771545d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:16:05.052614 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e063a6-cc0f-4e2f-9137-8dc3776141b7 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:05.069243 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 554/1107] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:16:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:16:05.088249 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d65dc-3fd3-4c3f-833e-1b528ca9caa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:05.088859 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d65dc-3fd3-4c3f-833e-1b528ca9caa5 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:05.089131 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d65dc-3fd3-4c3f-833e-1b528ca9caa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:05.089396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d65dc-3fd3-4c3f-833e-1b528ca9caa5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:05.090115 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d65dc-3fd3-4c3f-833e-1b528ca9caa5 None admin] RBAC: Adding query filter params (role.id=3631d69e831a474b807aeca180b214dc, scope.project.id=9f4373dfb5f9425eb73669e24d18a08f, user.id=cd7b11f479a54adc8d9b8641859b07a1) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:16:05.090417 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d65dc-3fd3-4c3f-833e-1b528ca9caa5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:05.091831 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d65dc-3fd3-4c3f-833e-1b528ca9caa5 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:05.097629 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 554/1108] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:16:04 2022] GET /identity/v3/role_assignments?user.id=cd7b11f479a54adc8d9b8641859b07a1&scope.project.id=9f4373dfb5f9425eb73669e24d18a08f&role.id=3631d69e831a474b807aeca180b214dc => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:05.344227 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85068fed-30f7-4f2e-8a33-244d771545d2 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:05.449823 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85068fed-30f7-4f2e-8a33-244d771545d2 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:05.488824 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b7cc25-a8d7-4e03-b726-118fb43ef592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:05.489463 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7cc25-a8d7-4e03-b726-118fb43ef592 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:05.489755 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7cc25-a8d7-4e03-b726-118fb43ef592 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:05.490023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7cc25-a8d7-4e03-b726-118fb43ef592 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:05.491234 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b7cc25-a8d7-4e03-b726-118fb43ef592 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:05.492752 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b7cc25-a8d7-4e03-b726-118fb43ef592 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:05.494347 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 555/1109] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:16:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:05.500453 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d65dc-3fd3-4c3f-833e-1b528ca9caa5 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:05.605090 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:05.608402 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d65dc-3fd3-4c3f-833e-1b528ca9caa5 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:05.643905 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ab48d-0cc6-4b6a-ab1f-2dea8f7a389a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:05.644586 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ab48d-0cc6-4b6a-ab1f-2dea8f7a389a None admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:05.644865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ab48d-0cc6-4b6a-ab1f-2dea8f7a389a None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:05.645211 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ab48d-0cc6-4b6a-ab1f-2dea8f7a389a None admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:05.645945 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ab48d-0cc6-4b6a-ab1f-2dea8f7a389a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:05.647372 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ab48d-0cc6-4b6a-ab1f-2dea8f7a389a None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:05.668688 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 555/1110] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Aug 23 04:16:05 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:16:07.593432 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4db43b-85ae-4787-93c5-0630cbd1b7a2 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:07.593983 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4db43b-85ae-4787-93c5-0630cbd1b7a2 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:07.593983 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4db43b-85ae-4787-93c5-0630cbd1b7a2 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:07.595302 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 556/1111] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:16:07.601745 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8bb47-f8b8-4158-b2df-9fb0a52dd343 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:07.602484 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8bb47-f8b8-4158-b2df-9fb0a52dd343 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:07.602636 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8bb47-f8b8-4158-b2df-9fb0a52dd343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:07.632207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-e2d8bb47-f8b8-4158-b2df-9fb0a52dd343 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:16:07.633391 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-e2d8bb47-f8b8-4158-b2df-9fb0a52dd343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:16:07.704050 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 556/1112] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:16:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:16:07.972655 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4db43b-85ae-4787-93c5-0630cbd1b7a2 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:07.977338 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:08.086916 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4db43b-85ae-4787-93c5-0630cbd1b7a2 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:08.123260 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ab834-b376-44ae-9222-6932a22561d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:08.123885 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ab834-b376-44ae-9222-6932a22561d4 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:08.124348 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ab834-b376-44ae-9222-6932a22561d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:08.124611 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ab834-b376-44ae-9222-6932a22561d4 None admin] PATH_INFO: `/v3/roles/4207a743edc04b0aba9197fe3554b2e3` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:08.126039 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ab834-b376-44ae-9222-6932a22561d4 None admin] RBAC: Authorizing `identity:get_role(role_id=4207a743edc04b0aba9197fe3554b2e3)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:08.127419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ab834-b376-44ae-9222-6932a22561d4 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:08.128782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 557/1113] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:16:07 2022] GET /identity/v3/roles/4207a743edc04b0aba9197fe3554b2e3 => generated 223 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:08.135564 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8bb47-f8b8-4158-b2df-9fb0a52dd343 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:08.227146 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:08.244345 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8bb47-f8b8-4158-b2df-9fb0a52dd343 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:08.284872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f0ac5-6ac7-47bf-bcf2-553eb224c9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:08.285988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f0ac5-6ac7-47bf-bcf2-553eb224c9e2 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:08.286396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f0ac5-6ac7-47bf-bcf2-553eb224c9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:08.286502 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f0ac5-6ac7-47bf-bcf2-553eb224c9e2 None admin] PATH_INFO: `/v3/users/bb7b3d747b544f4f87a45047381b126e` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:08.300597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f0ac5-6ac7-47bf-bcf2-553eb224c9e2 None admin] RBAC: Authorizing `identity:get_user(user_id=bb7b3d747b544f4f87a45047381b126e)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:08.302052 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f0ac5-6ac7-47bf-bcf2-553eb224c9e2 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:08.303658 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 557/1114] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:16:08 2022] GET /identity/v3/users/bb7b3d747b544f4f87a45047381b126e => generated 286 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:08.309908 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ab834-b376-44ae-9222-6932a22561d4 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:08.386205 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:08.417081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ab834-b376-44ae-9222-6932a22561d4 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:08.451869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0eff8-c534-42b4-80cf-9a67a635b97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:08.452493 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0eff8-c534-42b4-80cf-9a67a635b97d None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:08.452771 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0eff8-c534-42b4-80cf-9a67a635b97d None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:08.453037 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0eff8-c534-42b4-80cf-9a67a635b97d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:08.523224 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0eff8-c534-42b4-80cf-9a67a635b97d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:08.524635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0eff8-c534-42b4-80cf-9a67a635b97d None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:08.600224 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:08.637789 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 558/1115] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:16:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 04:16:08.644012 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f0ac5-6ac7-47bf-bcf2-553eb224c9e2 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:08.751245 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f0ac5-6ac7-47bf-bcf2-553eb224c9e2 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:08.787050 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2fd3b-5af5-4550-b946-5ed65c92b4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:08.787605 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2fd3b-5af5-4550-b946-5ed65c92b4f5 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:08.787880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2fd3b-5af5-4550-b946-5ed65c92b4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:08.788177 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2fd3b-5af5-4550-b946-5ed65c92b4f5 None admin] PATH_INFO: `/v3/projects/5fbe3eccfc48472693f4466cbfd55615` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:08.789420 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2fd3b-5af5-4550-b946-5ed65c92b4f5 None admin] RBAC: Authorizing `identity:get_project(project_id=5fbe3eccfc48472693f4466cbfd55615)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:08.790845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2fd3b-5af5-4550-b946-5ed65c92b4f5 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:08.792492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 558/1116] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:16:08 2022] GET /identity/v3/projects/5fbe3eccfc48472693f4466cbfd55615 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:08.799427 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0eff8-c534-42b4-80cf-9a67a635b97d None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:08.928420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0eff8-c534-42b4-80cf-9a67a635b97d None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:08.971282 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be1a05-e2f7-42e6-b4d0-67a13c9bc1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:08.971882 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be1a05-e2f7-42e6-b4d0-67a13c9bc1e2 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:08.972188 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be1a05-e2f7-42e6-b4d0-67a13c9bc1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:08.972458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be1a05-e2f7-42e6-b4d0-67a13c9bc1e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:08.973465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be1a05-e2f7-42e6-b4d0-67a13c9bc1e2 None admin] RBAC: Adding query filter params (role.id=4207a743edc04b0aba9197fe3554b2e3, scope.project.id=5fbe3eccfc48472693f4466cbfd55615, user.id=bb7b3d747b544f4f87a45047381b126e) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:16:08.973465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be1a05-e2f7-42e6-b4d0-67a13c9bc1e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:08.974935 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be1a05-e2f7-42e6-b4d0-67a13c9bc1e2 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:08.980681 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 559/1117] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:16:08 2022] GET /identity/v3/role_assignments?user.id=bb7b3d747b544f4f87a45047381b126e&scope.project.id=5fbe3eccfc48472693f4466cbfd55615&role.id=4207a743edc04b0aba9197fe3554b2e3 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:10.831787 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df5eda-d422-4edf-88b2-a686d2853b90 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:10.833084 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df5eda-d422-4edf-88b2-a686d2853b90 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:10.833084 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df5eda-d422-4edf-88b2-a686d2853b90 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:10.834938 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 559/1118] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:16:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:16:10.841528 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47731f7-bcc0-4801-9d8f-da464ee625cb None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:10.841975 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47731f7-bcc0-4801-9d8f-da464ee625cb None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:10.842073 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47731f7-bcc0-4801-9d8f-da464ee625cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:10.870039 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-b47731f7-bcc0-4801-9d8f-da464ee625cb None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:16:10.871236 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-b47731f7-bcc0-4801-9d8f-da464ee625cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:16:10.904031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:10.950040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 560/1119] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:16:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:16:11.213536 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df5eda-d422-4edf-88b2-a686d2853b90 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:11.233954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:11.341718 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df5eda-d422-4edf-88b2-a686d2853b90 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:11.377718 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04b220-4be4-4b49-8ccf-021706b31e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:11.378344 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04b220-4be4-4b49-8ccf-021706b31e42 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:11.378620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04b220-4be4-4b49-8ccf-021706b31e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:11.378888 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04b220-4be4-4b49-8ccf-021706b31e42 None admin] PATH_INFO: `/v3/roles/4207a743edc04b0aba9197fe3554b2e3` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:11.380452 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff04b220-4be4-4b49-8ccf-021706b31e42 None admin] RBAC: Authorizing `identity:get_role(role_id=4207a743edc04b0aba9197fe3554b2e3)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:11.381821 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff04b220-4be4-4b49-8ccf-021706b31e42 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:11.383151 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 560/1120] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:16:11 2022] GET /identity/v3/roles/4207a743edc04b0aba9197fe3554b2e3 => generated 223 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:11.392014 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47731f7-bcc0-4801-9d8f-da464ee625cb None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:11.508167 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47731f7-bcc0-4801-9d8f-da464ee625cb None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:11.544406 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e948c08-b65a-4d42-8c0c-b638a724fe0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:11.545040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e948c08-b65a-4d42-8c0c-b638a724fe0f None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:11.545316 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e948c08-b65a-4d42-8c0c-b638a724fe0f None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:11.545586 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e948c08-b65a-4d42-8c0c-b638a724fe0f None admin] PATH_INFO: `/v3/users/bb7b3d747b544f4f87a45047381b126e` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:11.546825 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e948c08-b65a-4d42-8c0c-b638a724fe0f None admin] RBAC: Authorizing `identity:get_user(user_id=bb7b3d747b544f4f87a45047381b126e)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:11.548298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e948c08-b65a-4d42-8c0c-b638a724fe0f None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:11.549894 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 561/1121] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:16:11 2022] GET /identity/v3/users/bb7b3d747b544f4f87a45047381b126e => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:11.556893 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04b220-4be4-4b49-8ccf-021706b31e42 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:11.648415 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:11.669525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04b220-4be4-4b49-8ccf-021706b31e42 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:11.705234 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25c738-2071-4b65-bdf9-0bc4d7936e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:11.705850 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25c738-2071-4b65-bdf9-0bc4d7936e11 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:11.706125 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25c738-2071-4b65-bdf9-0bc4d7936e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:11.706459 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25c738-2071-4b65-bdf9-0bc4d7936e11 None admin] PATH_INFO: `/v3/projects/5fbe3eccfc48472693f4466cbfd55615` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:11.707850 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25c738-2071-4b65-bdf9-0bc4d7936e11 None admin] RBAC: Authorizing `identity:get_project(project_id=5fbe3eccfc48472693f4466cbfd55615)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:11.709254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25c738-2071-4b65-bdf9-0bc4d7936e11 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:11.710871 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 561/1122] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:16:11 2022] GET /identity/v3/projects/5fbe3eccfc48472693f4466cbfd55615 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:11.717404 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e948c08-b65a-4d42-8c0c-b638a724fe0f None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:11.754765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:11.824447 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e948c08-b65a-4d42-8c0c-b638a724fe0f None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:11.860002 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd87c4-b430-4ca8-bfa1-b82ce7135de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:11.860600 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd87c4-b430-4ca8-bfa1-b82ce7135de4 None admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:11.860903 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd87c4-b430-4ca8-bfa1-b82ce7135de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:11.861175 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd87c4-b430-4ca8-bfa1-b82ce7135de4 None admin] PATH_INFO: `/v3/projects/5fbe3eccfc48472693f4466cbfd55615/users/bb7b3d747b544f4f87a45047381b126e/roles/4207a743edc04b0aba9197fe3554b2e3` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:11.863445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd87c4-b430-4ca8-bfa1-b82ce7135de4 None admin] RBAC: Authorizing `identity:create_grant(project_id=5fbe3eccfc48472693f4466cbfd55615, user_id=bb7b3d747b544f4f87a45047381b126e, role_id=4207a743edc04b0aba9197fe3554b2e3)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:11.865179 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd87c4-b430-4ca8-bfa1-b82ce7135de4 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:11.880904 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 562/1123] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Aug 23 04:16:11 2022] PUT /identity/v3/projects/5fbe3eccfc48472693f4466cbfd55615/users/bb7b3d747b544f4f87a45047381b126e/roles/4207a743edc04b0aba9197fe3554b2e3 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:16:13.758741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1754ac0-8cae-4124-b872-39cee5b31cd8 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:13.759583 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1754ac0-8cae-4124-b872-39cee5b31cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:13.759583 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1754ac0-8cae-4124-b872-39cee5b31cd8 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:13.761675 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 562/1124] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:16:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:16:13.770897 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2f8ea-8a2d-4e16-985e-3e88388af499 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:13.771324 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2f8ea-8a2d-4e16-985e-3e88388af499 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:13.771454 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2f8ea-8a2d-4e16-985e-3e88388af499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:13.800487 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-60b2f8ea-8a2d-4e16-985e-3e88388af499 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:16:13.801962 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-60b2f8ea-8a2d-4e16-985e-3e88388af499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:16:13.883036 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 563/1125] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:16:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:16:14.151009 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1754ac0-8cae-4124-b872-39cee5b31cd8 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:14.272662 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1754ac0-8cae-4124-b872-39cee5b31cd8 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:14.317474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64710ff5-536f-4bf8-a0db-79770b40b397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:14.318103 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64710ff5-536f-4bf8-a0db-79770b40b397 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:14.318376 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64710ff5-536f-4bf8-a0db-79770b40b397 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:14.318657 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64710ff5-536f-4bf8-a0db-79770b40b397 None admin] PATH_INFO: `/v3/roles/4207a743edc04b0aba9197fe3554b2e3` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:14.320116 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64710ff5-536f-4bf8-a0db-79770b40b397 None admin] RBAC: Authorizing `identity:get_role(role_id=4207a743edc04b0aba9197fe3554b2e3)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:14.321479 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64710ff5-536f-4bf8-a0db-79770b40b397 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:14.322820 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 563/1126] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:16:14 2022] GET /identity/v3/roles/4207a743edc04b0aba9197fe3554b2e3 => generated 223 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:14.331894 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2f8ea-8a2d-4e16-985e-3e88388af499 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:14.359313 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:14.437611 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2f8ea-8a2d-4e16-985e-3e88388af499 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:14.482496 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d5c0a-f3ed-4bd6-81e0-80f3edbe9f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:14.483452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d5c0a-f3ed-4bd6-81e0-80f3edbe9f28 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:14.483855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d5c0a-f3ed-4bd6-81e0-80f3edbe9f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:14.484299 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d5c0a-f3ed-4bd6-81e0-80f3edbe9f28 None admin] PATH_INFO: `/v3/users/bb7b3d747b544f4f87a45047381b126e` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:14.486139 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3d5c0a-f3ed-4bd6-81e0-80f3edbe9f28 None admin] RBAC: Authorizing `identity:get_user(user_id=bb7b3d747b544f4f87a45047381b126e)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:14.488523 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3d5c0a-f3ed-4bd6-81e0-80f3edbe9f28 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:14.490425 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 564/1127] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:16:14 2022] GET /identity/v3/users/bb7b3d747b544f4f87a45047381b126e => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:16:14.496488 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64710ff5-536f-4bf8-a0db-79770b40b397 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:14.515402 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:14.619406 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64710ff5-536f-4bf8-a0db-79770b40b397 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:14.660452 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e2b83-5e4f-4030-9e6b-0b5ddcd57697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:14.661081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e2b83-5e4f-4030-9e6b-0b5ddcd57697 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:14.661333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e2b83-5e4f-4030-9e6b-0b5ddcd57697 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:14.661632 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e2b83-5e4f-4030-9e6b-0b5ddcd57697 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:14.735635 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544e2b83-5e4f-4030-9e6b-0b5ddcd57697 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:14.737045 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544e2b83-5e4f-4030-9e6b-0b5ddcd57697 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:14.754081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:14.848359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 564/1128] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:16:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 04:16:14.858178 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d5c0a-f3ed-4bd6-81e0-80f3edbe9f28 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:14.954974 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:14.965471 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d5c0a-f3ed-4bd6-81e0-80f3edbe9f28 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:15.000780 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c67f1f0-9310-474f-884c-fb85581efef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:15.001335 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67f1f0-9310-474f-884c-fb85581efef0 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:15.001606 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67f1f0-9310-474f-884c-fb85581efef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:15.001872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67f1f0-9310-474f-884c-fb85581efef0 None admin] PATH_INFO: `/v3/projects/5fbe3eccfc48472693f4466cbfd55615` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:15.003022 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c67f1f0-9310-474f-884c-fb85581efef0 None admin] RBAC: Authorizing `identity:get_project(project_id=5fbe3eccfc48472693f4466cbfd55615)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:15.004548 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c67f1f0-9310-474f-884c-fb85581efef0 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:15.006299 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 565/1129] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:16:14 2022] GET /identity/v3/projects/5fbe3eccfc48472693f4466cbfd55615 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:15.012996 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e2b83-5e4f-4030-9e6b-0b5ddcd57697 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:15.124705 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e2b83-5e4f-4030-9e6b-0b5ddcd57697 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:15.141544 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:15.160563 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7e654-3b13-4d13-afc4-12919c4a2959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:15.161221 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7e654-3b13-4d13-afc4-12919c4a2959 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:15.161494 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7e654-3b13-4d13-afc4-12919c4a2959 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:15.161761 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7e654-3b13-4d13-afc4-12919c4a2959 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:15.162481 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7e654-3b13-4d13-afc4-12919c4a2959 None admin] RBAC: Adding query filter params (role.id=4207a743edc04b0aba9197fe3554b2e3, scope.project.id=5fbe3eccfc48472693f4466cbfd55615, user.id=bb7b3d747b544f4f87a45047381b126e) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:16:15.162785 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7e654-3b13-4d13-afc4-12919c4a2959 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:15.164235 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7e654-3b13-4d13-afc4-12919c4a2959 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:15.170036 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 565/1130] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:16:15 2022] GET /identity/v3/role_assignments?user.id=bb7b3d747b544f4f87a45047381b126e&scope.project.id=5fbe3eccfc48472693f4466cbfd55615&role.id=4207a743edc04b0aba9197fe3554b2e3 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:16:17.241225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869b75f-8a7c-4bc7-b111-e0b0d589066d None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:17.241826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869b75f-8a7c-4bc7-b111-e0b0d589066d None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:17.241826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869b75f-8a7c-4bc7-b111-e0b0d589066d None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:17.243134 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 566/1131] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:16:17.249507 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656c118-95b7-44cb-97e8-4574c81abd0c None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:17.250266 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656c118-95b7-44cb-97e8-4574c81abd0c None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:17.250412 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656c118-95b7-44cb-97e8-4574c81abd0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:17.271841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-8656c118-95b7-44cb-97e8-4574c81abd0c None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:16:17.273074 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-8656c118-95b7-44cb-97e8-4574c81abd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:16:17.347260 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 566/1132] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:16:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:16:17.611313 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e869b75f-8a7c-4bc7-b111-e0b0d589066d None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:17.732341 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e869b75f-8a7c-4bc7-b111-e0b0d589066d None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:17.768237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb2ab4b-d9ad-43ad-a435-29b67de689f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:17.768870 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2ab4b-d9ad-43ad-a435-29b67de689f7 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:17.769145 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2ab4b-d9ad-43ad-a435-29b67de689f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:17.769415 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2ab4b-d9ad-43ad-a435-29b67de689f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:17.771083 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb2ab4b-d9ad-43ad-a435-29b67de689f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:17.772583 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb2ab4b-d9ad-43ad-a435-29b67de689f7 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:17.774178 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 567/1133] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:16:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:17.780457 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656c118-95b7-44cb-97e8-4574c81abd0c None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:17.881461 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656c118-95b7-44cb-97e8-4574c81abd0c None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:17.891668 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:17.923091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3cc64-8738-4394-9c71-4d8e18983335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:17.923688 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3cc64-8738-4394-9c71-4d8e18983335 None admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:17.923955 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3cc64-8738-4394-9c71-4d8e18983335 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:17.924256 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3cc64-8738-4394-9c71-4d8e18983335 None admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:17.924888 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3cc64-8738-4394-9c71-4d8e18983335 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:17.926404 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3cc64-8738-4394-9c71-4d8e18983335 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:17.957646 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 567/1134] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Aug 23 04:16:17 2022] POST /identity/v3/projects => generated 311 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:16:19.873670 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72560e-0172-4a44-8d45-7eafe15574cf None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:19.874217 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72560e-0172-4a44-8d45-7eafe15574cf None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:19.874217 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72560e-0172-4a44-8d45-7eafe15574cf None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:19.875453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 568/1135] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:16:19.881911 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bdaadb-7ae6-43cd-8eb8-2f4b9beecf36 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:19.882613 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bdaadb-7ae6-43cd-8eb8-2f4b9beecf36 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:19.882759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bdaadb-7ae6-43cd-8eb8-2f4b9beecf36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:19.906433 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-52bdaadb-7ae6-43cd-8eb8-2f4b9beecf36 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:16:19.907627 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-52bdaadb-7ae6-43cd-8eb8-2f4b9beecf36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:16:19.977403 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 568/1136] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:16:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:16:20.244650 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e72560e-0172-4a44-8d45-7eafe15574cf None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:20.277647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:20.354553 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e72560e-0172-4a44-8d45-7eafe15574cf None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:20.391242 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cce92-9d68-4ff6-a969-c694055b3ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:20.391856 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cce92-9d68-4ff6-a969-c694055b3ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:20.392156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cce92-9d68-4ff6-a969-c694055b3ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:20.392422 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cce92-9d68-4ff6-a969-c694055b3ff9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:20.393569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9cce92-9d68-4ff6-a969-c694055b3ff9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:20.395085 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9cce92-9d68-4ff6-a969-c694055b3ff9 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:20.396682 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 569/1137] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:16:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:20.402705 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bdaadb-7ae6-43cd-8eb8-2f4b9beecf36 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:20.501371 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:20.511695 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bdaadb-7ae6-43cd-8eb8-2f4b9beecf36 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:20.546419 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53fca4-6dc0-4f92-91a5-c9fa478c07ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:20.547013 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53fca4-6dc0-4f92-91a5-c9fa478c07ad None admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:20.547270 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53fca4-6dc0-4f92-91a5-c9fa478c07ad None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:20.547535 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53fca4-6dc0-4f92-91a5-c9fa478c07ad None admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:20.548201 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca53fca4-6dc0-4f92-91a5-c9fa478c07ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:20.549632 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca53fca4-6dc0-4f92-91a5-c9fa478c07ad None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:20.577906 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 569/1138] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Aug 23 04:16:20 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:16:22.496109 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d2a32-3538-42a3-9ba3-35f9b1d65414 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:22.496651 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d2a32-3538-42a3-9ba3-35f9b1d65414 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:22.496651 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d2a32-3538-42a3-9ba3-35f9b1d65414 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:22.498010 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 570/1139] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:16:22.503385 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058ead4-3c4e-4c86-b246-92d1fbcd52d4 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:22.503792 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058ead4-3c4e-4c86-b246-92d1fbcd52d4 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:22.503950 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058ead4-3c4e-4c86-b246-92d1fbcd52d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:22.522895 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-1058ead4-3c4e-4c86-b246-92d1fbcd52d4 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:16:22.524216 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-1058ead4-3c4e-4c86-b246-92d1fbcd52d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:16:22.596830 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 570/1140] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:16:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:16:22.864608 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d2a32-3538-42a3-9ba3-35f9b1d65414 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:22.979851 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:22.987784 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d2a32-3538-42a3-9ba3-35f9b1d65414 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:23.023422 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa875ba-51d3-47e6-9aa2-fc2aedd14a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:23.024053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa875ba-51d3-47e6-9aa2-fc2aedd14a7b None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:23.024333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa875ba-51d3-47e6-9aa2-fc2aedd14a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:23.024595 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa875ba-51d3-47e6-9aa2-fc2aedd14a7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:23.035394 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa875ba-51d3-47e6-9aa2-fc2aedd14a7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:16:23.040480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-1aa875ba-51d3-47e6-9aa2-fc2aedd14a7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:16:23.040480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:16:23.040480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:16:23.040480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:16:23.040480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:16:23.040480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:16:23.040480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:16:23.040480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:16:23.040480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:16:23.040480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:16:23.040480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:16:23.040480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:16:23.040480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:16:23.040480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:16:23.040480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:16:23.040480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:16:23.041569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:16:23.041569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:16:23.041569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:16:23.041569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:16:23.041569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:16:23.041569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:16:23.041569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:16:23.041569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:16:23.041569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:16:23.041569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:16:23.041569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:16:23.041569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:16:23.041569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:16:23.041569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:16:23.041569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:16:23.041569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:16:23.042648 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:16:23.042648 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:16:23.042648 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:16:23.042648 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:16:23.042648 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:16:23.042648 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:16:23.042648 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:16:23.042648 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:16:23.042648 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:16:23.042648 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:16:23.042648 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:16:23.042648 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:16:23.042648 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 571/1141] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:16:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:16:23.049633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1058ead4-3c4e-4c86-b246-92d1fbcd52d4 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:23.135705 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:23.158566 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1058ead4-3c4e-4c86-b246-92d1fbcd52d4 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:23.194776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f247d1c-416c-447f-afa8-5ee487fa4611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:23.195372 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f247d1c-416c-447f-afa8-5ee487fa4611 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:23.195650 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f247d1c-416c-447f-afa8-5ee487fa4611 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:23.195946 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f247d1c-416c-447f-afa8-5ee487fa4611 None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:23.196606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f247d1c-416c-447f-afa8-5ee487fa4611 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:16:23.196910 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f247d1c-416c-447f-afa8-5ee487fa4611 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:23.198276 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f247d1c-416c-447f-afa8-5ee487fa4611 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:23.207882 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 571/1142] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:16:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:16:23.218034 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa875ba-51d3-47e6-9aa2-fc2aedd14a7b None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:23.330523 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa875ba-51d3-47e6-9aa2-fc2aedd14a7b None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:23.366478 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62047dd-2b43-445b-b262-73df1949679f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:23.367089 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62047dd-2b43-445b-b262-73df1949679f None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:23.367367 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62047dd-2b43-445b-b262-73df1949679f None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:23.367637 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62047dd-2b43-445b-b262-73df1949679f None admin] PATH_INFO: `/v3/users/a011198ac2744509af0ed27bac497370` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:23.381491 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62047dd-2b43-445b-b262-73df1949679f None admin] RBAC: Authorizing `identity:get_user(user_id=a011198ac2744509af0ed27bac497370)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:23.382937 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62047dd-2b43-445b-b262-73df1949679f None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:23.384552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 572/1143] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:16:23 2022] GET /identity/v3/users/a011198ac2744509af0ed27bac497370 => generated 286 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:23.391615 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f247d1c-416c-447f-afa8-5ee487fa4611 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:23.502635 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f247d1c-416c-447f-afa8-5ee487fa4611 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:23.538453 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e6fa4-fc00-4549-9ef1-8c36bc494af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:23.539031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e6fa4-fc00-4549-9ef1-8c36bc494af4 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:23.539351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e6fa4-fc00-4549-9ef1-8c36bc494af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:23.539624 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e6fa4-fc00-4549-9ef1-8c36bc494af4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:23.551422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:23.614621 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20e6fa4-fc00-4549-9ef1-8c36bc494af4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:23.616107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20e6fa4-fc00-4549-9ef1-8c36bc494af4 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:23.717723 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 572/1144] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:16:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 04:16:23.724629 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62047dd-2b43-445b-b262-73df1949679f None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:23.746667 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:23.835259 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62047dd-2b43-445b-b262-73df1949679f None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:23.870304 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89307d7b-b315-4582-b7d7-d32e0c7029ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:23.870913 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89307d7b-b315-4582-b7d7-d32e0c7029ce None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:23.871186 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89307d7b-b315-4582-b7d7-d32e0c7029ce None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:23.871456 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89307d7b-b315-4582-b7d7-d32e0c7029ce None admin] PATH_INFO: `/v3/projects/6e63b5baf1794154a40ea8a7b2f99e1f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:23.872619 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89307d7b-b315-4582-b7d7-d32e0c7029ce None admin] RBAC: Authorizing `identity:get_project(project_id=6e63b5baf1794154a40ea8a7b2f99e1f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:23.874051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89307d7b-b315-4582-b7d7-d32e0c7029ce None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:23.875734 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 573/1145] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:16:23 2022] GET /identity/v3/projects/6e63b5baf1794154a40ea8a7b2f99e1f => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:23.882214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e6fa4-fc00-4549-9ef1-8c36bc494af4 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:23.926909 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:23.986332 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e6fa4-fc00-4549-9ef1-8c36bc494af4 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:24.020927 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75556716-0b73-4648-bda4-1fa8105f6102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:24.021516 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75556716-0b73-4648-bda4-1fa8105f6102 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:24.021795 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75556716-0b73-4648-bda4-1fa8105f6102 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:24.022056 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75556716-0b73-4648-bda4-1fa8105f6102 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:24.022774 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75556716-0b73-4648-bda4-1fa8105f6102 None admin] RBAC: Adding query filter params (role.id=52b31061b72c45ccb76dd26bf31f8d3f, scope.project.id=6e63b5baf1794154a40ea8a7b2f99e1f, user.id=a011198ac2744509af0ed27bac497370) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:16:24.023068 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75556716-0b73-4648-bda4-1fa8105f6102 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:24.024520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75556716-0b73-4648-bda4-1fa8105f6102 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:24.030108 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 573/1146] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:16:23 2022] GET /identity/v3/role_assignments?user.id=a011198ac2744509af0ed27bac497370&scope.project.id=6e63b5baf1794154a40ea8a7b2f99e1f&role.id=52b31061b72c45ccb76dd26bf31f8d3f => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:25.897652 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435afd64-b465-4977-aa02-3c66dd344c7f None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:25.898209 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435afd64-b465-4977-aa02-3c66dd344c7f None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:25.898209 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435afd64-b465-4977-aa02-3c66dd344c7f None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:25.899564 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 574/1147] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:16:25.905937 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd982ef2-c768-49d3-958f-545eccdd74a2 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:25.906670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd982ef2-c768-49d3-958f-545eccdd74a2 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:25.906809 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd982ef2-c768-49d3-958f-545eccdd74a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:25.933559 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-cd982ef2-c768-49d3-958f-545eccdd74a2 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:16:25.934739 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-cd982ef2-c768-49d3-958f-545eccdd74a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:16:26.007557 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 574/1148] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:16:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:16:26.278015 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435afd64-b465-4977-aa02-3c66dd344c7f None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:26.381524 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:26.402729 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435afd64-b465-4977-aa02-3c66dd344c7f None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:26.446046 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb87031-8cf3-4bd2-8d86-7ef6a4c3c3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:26.447037 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb87031-8cf3-4bd2-8d86-7ef6a4c3c3cd None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:26.447499 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb87031-8cf3-4bd2-8d86-7ef6a4c3c3cd None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:26.447974 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb87031-8cf3-4bd2-8d86-7ef6a4c3c3cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:26.463598 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb87031-8cf3-4bd2-8d86-7ef6a4c3c3cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:16:26.468298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-2cb87031-8cf3-4bd2-8d86-7ef6a4c3c3cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:16:26.468298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:16:26.468298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:16:26.468298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:16:26.468298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:16:26.468298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:16:26.468298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:16:26.468298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:16:26.468298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:16:26.468298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:16:26.468298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:16:26.468298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:16:26.468298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:16:26.468298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:16:26.468298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:16:26.468298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:16:26.469432 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:16:26.469432 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:16:26.469432 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:16:26.469432 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:16:26.469432 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:16:26.469432 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:16:26.469432 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:16:26.469432 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:16:26.469432 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:16:26.469432 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:16:26.469432 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:16:26.469432 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:16:26.469432 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:16:26.469432 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:16:26.469432 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:16:26.469432 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:16:26.470420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:16:26.470420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:16:26.470420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:16:26.470420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:16:26.470420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:16:26.470420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:16:26.470420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:16:26.470420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:16:26.470420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:16:26.470420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:16:26.470420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:16:26.470420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:16:26.470420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 575/1149] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:16:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:16:26.476694 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd982ef2-c768-49d3-958f-545eccdd74a2 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:26.520261 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:26.609406 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd982ef2-c768-49d3-958f-545eccdd74a2 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:26.645760 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4476eb0-243e-4eae-8c8a-77d477e9f8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:26.646376 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4476eb0-243e-4eae-8c8a-77d477e9f8c9 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:26.646651 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4476eb0-243e-4eae-8c8a-77d477e9f8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:26.646919 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4476eb0-243e-4eae-8c8a-77d477e9f8c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:26.647589 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4476eb0-243e-4eae-8c8a-77d477e9f8c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:16:26.647888 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4476eb0-243e-4eae-8c8a-77d477e9f8c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:26.649286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4476eb0-243e-4eae-8c8a-77d477e9f8c9 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:26.657506 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 575/1150] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:16:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:26.664120 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb87031-8cf3-4bd2-8d86-7ef6a4c3c3cd None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:26.779358 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb87031-8cf3-4bd2-8d86-7ef6a4c3c3cd None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:26.819533 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d480b-36ed-42e4-b7ac-bbbe24058e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:26.820157 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d480b-36ed-42e4-b7ac-bbbe24058e9b None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:26.820465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d480b-36ed-42e4-b7ac-bbbe24058e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:26.820736 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d480b-36ed-42e4-b7ac-bbbe24058e9b None admin] PATH_INFO: `/v3/users/a011198ac2744509af0ed27bac497370` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:26.821910 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d480b-36ed-42e4-b7ac-bbbe24058e9b None admin] RBAC: Authorizing `identity:get_user(user_id=a011198ac2744509af0ed27bac497370)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:26.823364 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d480b-36ed-42e4-b7ac-bbbe24058e9b None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:26.824993 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 576/1151] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:16:26 2022] GET /identity/v3/users/a011198ac2744509af0ed27bac497370 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:26.831890 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4476eb0-243e-4eae-8c8a-77d477e9f8c9 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:26.947298 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:26.953496 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4476eb0-243e-4eae-8c8a-77d477e9f8c9 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:26.990002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ae3fe-4fa3-4bbf-99b2-65c2f09e65da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:26.990609 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ae3fe-4fa3-4bbf-99b2-65c2f09e65da None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:26.990888 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ae3fe-4fa3-4bbf-99b2-65c2f09e65da None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:26.991151 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ae3fe-4fa3-4bbf-99b2-65c2f09e65da None admin] PATH_INFO: `/v3/projects/6e63b5baf1794154a40ea8a7b2f99e1f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:26.992403 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ae3fe-4fa3-4bbf-99b2-65c2f09e65da None admin] RBAC: Authorizing `identity:get_project(project_id=6e63b5baf1794154a40ea8a7b2f99e1f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:26.993832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ae3fe-4fa3-4bbf-99b2-65c2f09e65da None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:26.995478 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 576/1152] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:16:26 2022] GET /identity/v3/projects/6e63b5baf1794154a40ea8a7b2f99e1f => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:27.005161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d480b-36ed-42e4-b7ac-bbbe24058e9b None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:27.018395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:27.110391 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d480b-36ed-42e4-b7ac-bbbe24058e9b None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:27.145384 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df3bfd-3b9a-45c7-8ecb-b5c9c4e94440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:27.146037 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df3bfd-3b9a-45c7-8ecb-b5c9c4e94440 None admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:27.146312 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df3bfd-3b9a-45c7-8ecb-b5c9c4e94440 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:27.146584 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df3bfd-3b9a-45c7-8ecb-b5c9c4e94440 None admin] PATH_INFO: `/v3/projects/6e63b5baf1794154a40ea8a7b2f99e1f/users/a011198ac2744509af0ed27bac497370/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:27.148595 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64df3bfd-3b9a-45c7-8ecb-b5c9c4e94440 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e63b5baf1794154a40ea8a7b2f99e1f, user_id=a011198ac2744509af0ed27bac497370, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:27.150030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64df3bfd-3b9a-45c7-8ecb-b5c9c4e94440 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:27.170025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 577/1153] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Aug 23 04:16:27 2022] PUT /identity/v3/projects/6e63b5baf1794154a40ea8a7b2f99e1f/users/a011198ac2744509af0ed27bac497370/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:16:29.023952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19778c28-25d5-4617-83bd-b2ae7af6dbdd None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:29.024514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19778c28-25d5-4617-83bd-b2ae7af6dbdd None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:29.024514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19778c28-25d5-4617-83bd-b2ae7af6dbdd None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:29.025859 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 577/1154] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:16:29.031265 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880140d6-d2b9-45a8-bb9f-e17cb2fb0dfc None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:29.031684 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880140d6-d2b9-45a8-bb9f-e17cb2fb0dfc None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:29.031790 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880140d6-d2b9-45a8-bb9f-e17cb2fb0dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:29.050629 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-880140d6-d2b9-45a8-bb9f-e17cb2fb0dfc None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:16:29.051941 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-880140d6-d2b9-45a8-bb9f-e17cb2fb0dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:16:29.125364 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:29.130769 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 578/1155] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:16:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:16:29.395725 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19778c28-25d5-4617-83bd-b2ae7af6dbdd None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:29.516061 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19778c28-25d5-4617-83bd-b2ae7af6dbdd None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:29.555797 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc0a66f-1bfb-4cda-8de0-68471aacc1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:29.556459 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0a66f-1bfb-4cda-8de0-68471aacc1d8 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:29.556737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0a66f-1bfb-4cda-8de0-68471aacc1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:29.557005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0a66f-1bfb-4cda-8de0-68471aacc1d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:29.568234 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc0a66f-1bfb-4cda-8de0-68471aacc1d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:16:29.572868 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-ecc0a66f-1bfb-4cda-8de0-68471aacc1d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:16:29.572868 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:16:29.572868 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:16:29.572868 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:16:29.572868 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:16:29.572868 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:16:29.572868 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:16:29.572868 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:16:29.572868 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:16:29.572868 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:16:29.572868 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:16:29.572868 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:16:29.572868 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:16:29.572868 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:16:29.572868 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:16:29.572868 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:16:29.573907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:16:29.573907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:16:29.573907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:16:29.573907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:16:29.573907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:16:29.573907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:16:29.573907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:16:29.573907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:16:29.573907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:16:29.573907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:16:29.573907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:16:29.573907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:16:29.573907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:16:29.573907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:16:29.573907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:16:29.573907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:16:29.574861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:16:29.574861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:16:29.574861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:16:29.574861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:16:29.574861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:16:29.574861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:16:29.574861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:16:29.574861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:16:29.574861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:16:29.574861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:16:29.574861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:16:29.574861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:16:29.574861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 578/1156] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:16:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:16:29.580960 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880140d6-d2b9-45a8-bb9f-e17cb2fb0dfc None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:29.684881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880140d6-d2b9-45a8-bb9f-e17cb2fb0dfc None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:29.720798 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbd3711-289d-4e8f-902c-fdee9b9bf0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:29.721389 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd3711-289d-4e8f-902c-fdee9b9bf0b4 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:29.721664 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd3711-289d-4e8f-902c-fdee9b9bf0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:29.721931 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd3711-289d-4e8f-902c-fdee9b9bf0b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:29.722594 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbd3711-289d-4e8f-902c-fdee9b9bf0b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:16:29.722908 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbd3711-289d-4e8f-902c-fdee9b9bf0b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:29.724458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbd3711-289d-4e8f-902c-fdee9b9bf0b4 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:29.732538 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 579/1157] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:16:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:16:29.740894 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc0a66f-1bfb-4cda-8de0-68471aacc1d8 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:29.770048 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:29.870779 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc0a66f-1bfb-4cda-8de0-68471aacc1d8 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:29.917405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577bd99f-3a27-493f-929e-a4889b31f26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:29.918035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bd99f-3a27-493f-929e-a4889b31f26e None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:29.918311 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bd99f-3a27-493f-929e-a4889b31f26e None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:29.918584 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bd99f-3a27-493f-929e-a4889b31f26e None admin] PATH_INFO: `/v3/users/a011198ac2744509af0ed27bac497370` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:29.919826 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577bd99f-3a27-493f-929e-a4889b31f26e None admin] RBAC: Authorizing `identity:get_user(user_id=a011198ac2744509af0ed27bac497370)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:29.921316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577bd99f-3a27-493f-929e-a4889b31f26e None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:29.922901 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 579/1158] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:16:29 2022] GET /identity/v3/users/a011198ac2744509af0ed27bac497370 => generated 286 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:29.929675 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbd3711-289d-4e8f-902c-fdee9b9bf0b4 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:29.999444 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:30.055033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbd3711-289d-4e8f-902c-fdee9b9bf0b4 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:30.091259 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd882bde-4b46-4522-a843-49ac95c57671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:30.091859 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd882bde-4b46-4522-a843-49ac95c57671 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:30.092164 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd882bde-4b46-4522-a843-49ac95c57671 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:30.092439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd882bde-4b46-4522-a843-49ac95c57671 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:30.166266 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd882bde-4b46-4522-a843-49ac95c57671 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:30.167724 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd882bde-4b46-4522-a843-49ac95c57671 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:30.236710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:30.282489 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 580/1159] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:16:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 04:16:30.290872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577bd99f-3a27-493f-929e-a4889b31f26e None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:30.395177 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:30.405961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577bd99f-3a27-493f-929e-a4889b31f26e None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:30.442965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb359949-ae34-465b-8381-d067eec7c957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:30.443577 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb359949-ae34-465b-8381-d067eec7c957 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:30.443850 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb359949-ae34-465b-8381-d067eec7c957 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:30.444137 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb359949-ae34-465b-8381-d067eec7c957 None admin] PATH_INFO: `/v3/projects/6e63b5baf1794154a40ea8a7b2f99e1f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:30.445350 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb359949-ae34-465b-8381-d067eec7c957 None admin] RBAC: Authorizing `identity:get_project(project_id=6e63b5baf1794154a40ea8a7b2f99e1f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:30.446770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb359949-ae34-465b-8381-d067eec7c957 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:30.448431 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 580/1160] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:16:30 2022] GET /identity/v3/projects/6e63b5baf1794154a40ea8a7b2f99e1f => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:30.455110 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd882bde-4b46-4522-a843-49ac95c57671 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:30.560414 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd882bde-4b46-4522-a843-49ac95c57671 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:30.598032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39274f4e-46d4-4672-a178-fdfae1b2f664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:30.598705 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39274f4e-46d4-4672-a178-fdfae1b2f664 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:30.598980 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39274f4e-46d4-4672-a178-fdfae1b2f664 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:30.599218 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39274f4e-46d4-4672-a178-fdfae1b2f664 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:30.599999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39274f4e-46d4-4672-a178-fdfae1b2f664 None admin] RBAC: Adding query filter params (role.id=52b31061b72c45ccb76dd26bf31f8d3f, scope.project.id=6e63b5baf1794154a40ea8a7b2f99e1f, user.id=a011198ac2744509af0ed27bac497370) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:16:30.600301 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39274f4e-46d4-4672-a178-fdfae1b2f664 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:30.601740 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39274f4e-46d4-4672-a178-fdfae1b2f664 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:30.607609 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 581/1161] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:16:30 2022] GET /identity/v3/role_assignments?user.id=a011198ac2744509af0ed27bac497370&scope.project.id=6e63b5baf1794154a40ea8a7b2f99e1f&role.id=52b31061b72c45ccb76dd26bf31f8d3f => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:32.525187 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8819aa-98fd-4a3c-b585-d162b51869fd None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:32.525737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8819aa-98fd-4a3c-b585-d162b51869fd None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:32.525737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8819aa-98fd-4a3c-b585-d162b51869fd None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:32.527078 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 581/1162] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:16:32.533526 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e21eaa-f202-4d6e-b453-6e827899988d None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:32.534274 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e21eaa-f202-4d6e-b453-6e827899988d None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:32.534420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e21eaa-f202-4d6e-b453-6e827899988d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:32.557686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-27e21eaa-f202-4d6e-b453-6e827899988d None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:16:32.558855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-27e21eaa-f202-4d6e-b453-6e827899988d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:16:32.582023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:32.630288 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 582/1163] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:16:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:16:32.891330 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8819aa-98fd-4a3c-b585-d162b51869fd None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:33.011956 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8819aa-98fd-4a3c-b585-d162b51869fd None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:33.046688 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f078ae-9205-4fc7-887c-5562a64758aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:33.047285 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f078ae-9205-4fc7-887c-5562a64758aa None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:33.047544 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f078ae-9205-4fc7-887c-5562a64758aa None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:33.047810 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f078ae-9205-4fc7-887c-5562a64758aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:33.076123 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:33.126675 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f078ae-9205-4fc7-887c-5562a64758aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:33.128987 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f078ae-9205-4fc7-887c-5562a64758aa None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:33.244210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 582/1164] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:16:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 04:16:33.251109 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e21eaa-f202-4d6e-b453-6e827899988d None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:33.367926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e21eaa-f202-4d6e-b453-6e827899988d None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:33.406434 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1be536-12a3-4930-b039-34529fffd49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:33.407050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1be536-12a3-4930-b039-34529fffd49d None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:33.407327 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1be536-12a3-4930-b039-34529fffd49d None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:33.407595 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1be536-12a3-4930-b039-34529fffd49d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:33.414573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1be536-12a3-4930-b039-34529fffd49d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:33.416051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1be536-12a3-4930-b039-34529fffd49d None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:33.427551 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-ee1be536-12a3-4930-b039-34529fffd49d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 23 04:16:33.427551 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:16:33.427551 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 23 04:16:33.427551 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 23 04:16:33.427551 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:16:33.427551 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:16:33.427551 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:16:33.427551 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:16:33.427551 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 23 04:16:33.427551 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:16:33.427551 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:16:33.427551 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:16:33.427551 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:16:33.427551 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:16:33.427551 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:16:33.427551 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:16:33.427551 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:16:33.428768 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:16:33.428768 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:16:33.428768 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:16:33.428768 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:16:33.428768 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:16:33.428768 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:16:33.428768 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:16:33.428768 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:16:33.428768 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:16:33.428768 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:16:33.428768 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:16:33.428768 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:16:33.428768 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:16:33.428768 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:16:33.428768 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:16:33.428768 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:16:33.429710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:16:33.429710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:16:33.429710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:16:33.429710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:16:33.429710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:16:33.429710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:16:33.429710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:16:33.429710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:16:33.429710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:16:33.429710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:16:33.429710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 23 04:16:33.429710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:16:33.429710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 23 04:16:33.429710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:16:33.429710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 583/1165] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Aug 23 04:16:33 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:16:33.435033 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f078ae-9205-4fc7-887c-5562a64758aa None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:33.498292 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:33.545281 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f078ae-9205-4fc7-887c-5562a64758aa None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:33.589764 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f195db4-7138-4903-ae1a-9e0f79857778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:33.590360 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f195db4-7138-4903-ae1a-9e0f79857778 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:33.590633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f195db4-7138-4903-ae1a-9e0f79857778 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:33.590900 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f195db4-7138-4903-ae1a-9e0f79857778 None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:33.591536 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f195db4-7138-4903-ae1a-9e0f79857778 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:16:33.591825 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f195db4-7138-4903-ae1a-9e0f79857778 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:33.593219 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f195db4-7138-4903-ae1a-9e0f79857778 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:33.600744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 583/1166] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Aug 23 04:16:33 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:35.487622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb28fe-46aa-445f-8abf-a6e1d3fcb4dc None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:35.488152 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb28fe-46aa-445f-8abf-a6e1d3fcb4dc None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:35.488152 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb28fe-46aa-445f-8abf-a6e1d3fcb4dc None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:35.489544 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 584/1167] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:16:35.495983 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe60b89-0e73-42f9-bbe0-f70b7c9516d6 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:35.496392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe60b89-0e73-42f9-bbe0-f70b7c9516d6 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:35.496527 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe60b89-0e73-42f9-bbe0-f70b7c9516d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:35.525181 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-abe60b89-0e73-42f9-bbe0-f70b7c9516d6 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:16:35.526353 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-abe60b89-0e73-42f9-bbe0-f70b7c9516d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:16:35.610215 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 584/1168] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:16:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:16:35.877990 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb28fe-46aa-445f-8abf-a6e1d3fcb4dc None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:35.902131 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:35.994208 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb28fe-46aa-445f-8abf-a6e1d3fcb4dc None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:36.029416 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3b133-253d-492c-a5ef-167ca8e7609a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:36.030011 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3b133-253d-492c-a5ef-167ca8e7609a None admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:36.030287 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3b133-253d-492c-a5ef-167ca8e7609a None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:36.030555 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3b133-253d-492c-a5ef-167ca8e7609a None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:36.031191 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e3b133-253d-492c-a5ef-167ca8e7609a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:36.032593 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e3b133-253d-492c-a5ef-167ca8e7609a None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:36.053355 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 585/1169] 10.222.0.18 () {60 vars in 1138 bytes} [Tue Aug 23 04:16:35 2022] POST /identity/v3/domains => generated 267 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:16:37.946856 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f7192-3f01-4712-878d-6dc0dd8aa384 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:37.947406 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f7192-3f01-4712-878d-6dc0dd8aa384 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:37.947406 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f7192-3f01-4712-878d-6dc0dd8aa384 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:37.948775 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 585/1170] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:16:37.955134 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7473393-d866-4ba1-96ed-6687785396a5 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:37.955815 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7473393-d866-4ba1-96ed-6687785396a5 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:37.956050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7473393-d866-4ba1-96ed-6687785396a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:37.982029 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-e7473393-d866-4ba1-96ed-6687785396a5 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:16:37.983223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-e7473393-d866-4ba1-96ed-6687785396a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:16:38.058382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 586/1171] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:16:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:16:38.320750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f7192-3f01-4712-878d-6dc0dd8aa384 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:38.381508 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:38.429444 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f7192-3f01-4712-878d-6dc0dd8aa384 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:38.470025 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d646e100-855b-48f8-bfb7-dcff8b40c7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:38.470700 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646e100-855b-48f8-bfb7-dcff8b40c7b3 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:38.470973 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646e100-855b-48f8-bfb7-dcff8b40c7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:38.471239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646e100-855b-48f8-bfb7-dcff8b40c7b3 None admin] PATH_INFO: `/v3/domains/cd7c04d390fc47ceab83bf89a210a697` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:38.482614 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d646e100-855b-48f8-bfb7-dcff8b40c7b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=cd7c04d390fc47ceab83bf89a210a697)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:38.484075 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d646e100-855b-48f8-bfb7-dcff8b40c7b3 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:38.485637 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 586/1172] 10.222.0.18 () {56 vars in 1183 bytes} [Tue Aug 23 04:16:38 2022] GET /identity/v3/domains/cd7c04d390fc47ceab83bf89a210a697 => generated 267 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:38.491885 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7473393-d866-4ba1-96ed-6687785396a5 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:38.500370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:38.604325 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7473393-d866-4ba1-96ed-6687785396a5 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:38.640213 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5940ff03-c6f1-4816-853e-2bb4062537b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:38.640813 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5940ff03-c6f1-4816-853e-2bb4062537b4 None admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:38.641084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5940ff03-c6f1-4816-853e-2bb4062537b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:38.641346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5940ff03-c6f1-4816-853e-2bb4062537b4 None admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:38.641981 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5940ff03-c6f1-4816-853e-2bb4062537b4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:38.643422 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5940ff03-c6f1-4816-853e-2bb4062537b4 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:38.676325 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 587/1173] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Aug 23 04:16:38 2022] POST /identity/v3/projects => generated 361 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:16:40.613846 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c889cf-a1a0-4572-a92d-b68da67bd3ba None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:40.614802 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c889cf-a1a0-4572-a92d-b68da67bd3ba None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:40.614802 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c889cf-a1a0-4572-a92d-b68da67bd3ba None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:40.617006 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 587/1174] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:16:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:16:40.622485 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167177ea-9645-42e3-bfe6-0d4a63f28d6a None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:40.622925 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167177ea-9645-42e3-bfe6-0d4a63f28d6a None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:40.623021 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167177ea-9645-42e3-bfe6-0d4a63f28d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:40.642117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-167177ea-9645-42e3-bfe6-0d4a63f28d6a None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:16:40.643291 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-167177ea-9645-42e3-bfe6-0d4a63f28d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:16:40.697564 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:40.714692 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 588/1175] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:16:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:16:40.985936 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c889cf-a1a0-4572-a92d-b68da67bd3ba None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:41.104344 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c889cf-a1a0-4572-a92d-b68da67bd3ba None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:41.117805 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:41.138401 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c8f20-21fd-42f8-8d4d-28bc45493e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:41.139145 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c8f20-21fd-42f8-8d4d-28bc45493e1b None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:41.139453 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c8f20-21fd-42f8-8d4d-28bc45493e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:41.139759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c8f20-21fd-42f8-8d4d-28bc45493e1b None admin] PATH_INFO: `/v3/domains/cd7c04d390fc47ceab83bf89a210a697` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:41.140959 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c8f20-21fd-42f8-8d4d-28bc45493e1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=cd7c04d390fc47ceab83bf89a210a697)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:41.142378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c8f20-21fd-42f8-8d4d-28bc45493e1b None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:41.143946 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 588/1176] 10.222.0.18 () {56 vars in 1183 bytes} [Tue Aug 23 04:16:40 2022] GET /identity/v3/domains/cd7c04d390fc47ceab83bf89a210a697 => generated 267 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:41.150050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167177ea-9645-42e3-bfe6-0d4a63f28d6a None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:41.278111 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167177ea-9645-42e3-bfe6-0d4a63f28d6a None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:41.314689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aff8d6-ad5d-41cc-b188-86c669978cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:41.315299 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aff8d6-ad5d-41cc-b188-86c669978cc8 None admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:41.315573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aff8d6-ad5d-41cc-b188-86c669978cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:41.315852 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aff8d6-ad5d-41cc-b188-86c669978cc8 None admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:41.316535 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aff8d6-ad5d-41cc-b188-86c669978cc8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:41.317991 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aff8d6-ad5d-41cc-b188-86c669978cc8 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:41.343879 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 589/1177] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Aug 23 04:16:41 2022] POST /identity/v3/users => generated 311 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:16:43.291104 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185b563-f14e-43ab-93b5-8964eeb385e7 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:43.291707 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185b563-f14e-43ab-93b5-8964eeb385e7 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:43.291707 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185b563-f14e-43ab-93b5-8964eeb385e7 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:43.293030 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 589/1178] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:16:43.299591 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896c4ac-adcf-49fd-b4da-a3a6cd415542 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:43.300381 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896c4ac-adcf-49fd-b4da-a3a6cd415542 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:43.300560 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896c4ac-adcf-49fd-b4da-a3a6cd415542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:43.326600 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-2896c4ac-adcf-49fd-b4da-a3a6cd415542 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:16:43.327808 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-2896c4ac-adcf-49fd-b4da-a3a6cd415542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:16:43.370977 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:43.401071 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 590/1179] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:16:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:16:43.672060 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185b563-f14e-43ab-93b5-8964eeb385e7 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:43.794403 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185b563-f14e-43ab-93b5-8964eeb385e7 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:43.831695 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46f9c8-ec63-409a-9486-3b6c1447310a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:43.832344 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46f9c8-ec63-409a-9486-3b6c1447310a None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:43.832620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46f9c8-ec63-409a-9486-3b6c1447310a None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:43.832897 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46f9c8-ec63-409a-9486-3b6c1447310a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:43.845600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46f9c8-ec63-409a-9486-3b6c1447310a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:16:43.850685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-cd46f9c8-ec63-409a-9486-3b6c1447310a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:16:43.850685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:16:43.850685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:16:43.850685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:16:43.850685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:16:43.850685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:16:43.850685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:16:43.850685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:16:43.850685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:16:43.850685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:16:43.850685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:16:43.850685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:16:43.850685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:16:43.850685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:16:43.850685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:16:43.850685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:16:43.851880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:16:43.851880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:16:43.851880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:16:43.851880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:16:43.851880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:16:43.851880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:16:43.851880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:16:43.851880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:16:43.851880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:16:43.851880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:16:43.851880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:16:43.851880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:16:43.851880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:16:43.851880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:16:43.851880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:16:43.851880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:16:43.852781 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:16:43.852781 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:16:43.852781 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:16:43.852781 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:16:43.852781 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:16:43.852781 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:16:43.852781 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:16:43.852781 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:16:43.852781 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:16:43.852781 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:16:43.852781 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:16:43.852781 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:16:43.852781 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 590/1180] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:16:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:16:43.861388 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2896c4ac-adcf-49fd-b4da-a3a6cd415542 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:43.969281 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2896c4ac-adcf-49fd-b4da-a3a6cd415542 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:44.004324 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d9200-d0d6-4227-b58f-e5a33c28c3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:44.004918 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d9200-d0d6-4227-b58f-e5a33c28c3ca None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:44.005191 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d9200-d0d6-4227-b58f-e5a33c28c3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:44.005457 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d9200-d0d6-4227-b58f-e5a33c28c3ca None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:44.006191 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05d9200-d0d6-4227-b58f-e5a33c28c3ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:16:44.006478 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05d9200-d0d6-4227-b58f-e5a33c28c3ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:44.007853 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05d9200-d0d6-4227-b58f-e5a33c28c3ca None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:44.016060 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 591/1181] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:16:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:44.022166 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46f9c8-ec63-409a-9486-3b6c1447310a None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:44.070588 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:44.130266 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46f9c8-ec63-409a-9486-3b6c1447310a None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:44.165509 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce26151-1ab0-4e6a-b386-5889df11fd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:44.166138 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce26151-1ab0-4e6a-b386-5889df11fd12 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:44.166416 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce26151-1ab0-4e6a-b386-5889df11fd12 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:44.166689 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce26151-1ab0-4e6a-b386-5889df11fd12 None admin] PATH_INFO: `/v3/users/695d62082a5b4d5486e74fb58cb0e05f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:44.180732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce26151-1ab0-4e6a-b386-5889df11fd12 None admin] RBAC: Authorizing `identity:get_user(user_id=695d62082a5b4d5486e74fb58cb0e05f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:44.182156 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce26151-1ab0-4e6a-b386-5889df11fd12 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:44.183750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 591/1182] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:16:44 2022] GET /identity/v3/users/695d62082a5b4d5486e74fb58cb0e05f => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:44.190505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d9200-d0d6-4227-b58f-e5a33c28c3ca None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:44.232661 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:44.307537 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d9200-d0d6-4227-b58f-e5a33c28c3ca None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:44.344495 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546b7e6-0616-4771-9704-111874ed2605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:44.345089 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546b7e6-0616-4771-9704-111874ed2605 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:44.345365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546b7e6-0616-4771-9704-111874ed2605 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:44.345639 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546b7e6-0616-4771-9704-111874ed2605 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:44.414347 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546b7e6-0616-4771-9704-111874ed2605 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:44.415799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546b7e6-0616-4771-9704-111874ed2605 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:44.451750 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:44.521126 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 592/1183] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:16:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 04:16:44.530702 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce26151-1ab0-4e6a-b386-5889df11fd12 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:44.641477 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:44.644748 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce26151-1ab0-4e6a-b386-5889df11fd12 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:44.683824 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa73a89-6ffe-40a0-b3e6-704d59ae9b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:44.684475 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa73a89-6ffe-40a0-b3e6-704d59ae9b32 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:44.684748 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa73a89-6ffe-40a0-b3e6-704d59ae9b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:44.685027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa73a89-6ffe-40a0-b3e6-704d59ae9b32 None admin] PATH_INFO: `/v3/projects/da9318cc9af34bb2a97df33186ea8977` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:44.686261 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa73a89-6ffe-40a0-b3e6-704d59ae9b32 None admin] RBAC: Authorizing `identity:get_project(project_id=da9318cc9af34bb2a97df33186ea8977)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:44.687699 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa73a89-6ffe-40a0-b3e6-704d59ae9b32 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:44.689366 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 592/1184] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:16:44 2022] GET /identity/v3/projects/da9318cc9af34bb2a97df33186ea8977 => generated 361 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:44.695840 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546b7e6-0616-4771-9704-111874ed2605 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:44.804168 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546b7e6-0616-4771-9704-111874ed2605 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:44.839643 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46173853-74f1-420e-985f-056f355efac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:44.840265 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46173853-74f1-420e-985f-056f355efac0 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:44.840534 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46173853-74f1-420e-985f-056f355efac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:44.840800 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46173853-74f1-420e-985f-056f355efac0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:44.841516 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46173853-74f1-420e-985f-056f355efac0 None admin] RBAC: Adding query filter params (role.id=52b31061b72c45ccb76dd26bf31f8d3f, scope.project.id=da9318cc9af34bb2a97df33186ea8977, user.id=695d62082a5b4d5486e74fb58cb0e05f) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:16:44.841819 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46173853-74f1-420e-985f-056f355efac0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:44.843235 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46173853-74f1-420e-985f-056f355efac0 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:44.846098 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:44.849183 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 593/1185] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:16:44 2022] GET /identity/v3/role_assignments?user.id=695d62082a5b4d5486e74fb58cb0e05f&scope.project.id=da9318cc9af34bb2a97df33186ea8977&role.id=52b31061b72c45ccb76dd26bf31f8d3f => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:46.731016 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23a9ad-97a9-465c-91a9-876d8c0b33cf None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:46.731541 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23a9ad-97a9-465c-91a9-876d8c0b33cf None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:46.731541 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23a9ad-97a9-465c-91a9-876d8c0b33cf None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:46.732912 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 593/1186] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:16:46.739215 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc560c-c592-4f2e-ac53-01fcda59f2f7 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:46.739977 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc560c-c592-4f2e-ac53-01fcda59f2f7 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:46.740261 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc560c-c592-4f2e-ac53-01fcda59f2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:46.760994 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-acbc560c-c592-4f2e-ac53-01fcda59f2f7 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:16:46.762176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-acbc560c-c592-4f2e-ac53-01fcda59f2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:16:46.832729 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 594/1187] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:16:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:16:47.098560 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23a9ad-97a9-465c-91a9-876d8c0b33cf None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:47.222614 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23a9ad-97a9-465c-91a9-876d8c0b33cf None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:47.260160 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69447132-cda1-483b-aa37-1fe605c154e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:47.260778 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69447132-cda1-483b-aa37-1fe605c154e8 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:47.261057 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69447132-cda1-483b-aa37-1fe605c154e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:47.261324 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69447132-cda1-483b-aa37-1fe605c154e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:47.271711 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69447132-cda1-483b-aa37-1fe605c154e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:16:47.276386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-69447132-cda1-483b-aa37-1fe605c154e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:16:47.276386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:16:47.276386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:16:47.276386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:16:47.276386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:16:47.276386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:16:47.276386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:16:47.276386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:16:47.276386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:16:47.276386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:16:47.276386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:16:47.276386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:16:47.276386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:16:47.276386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:16:47.276386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:16:47.276386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:16:47.277366 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:16:47.277366 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:16:47.277366 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:16:47.277366 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:16:47.277366 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:16:47.277366 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:16:47.277366 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:16:47.277366 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:16:47.277366 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:16:47.277366 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:16:47.277366 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:16:47.277366 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:16:47.277366 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:16:47.277366 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:16:47.277366 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:16:47.277366 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:16:47.278227 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:16:47.278227 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:16:47.278227 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:16:47.278227 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:16:47.278227 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:16:47.278227 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:16:47.278227 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:16:47.278227 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:16:47.278227 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:16:47.278227 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:16:47.278227 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:16:47.278227 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:16:47.278227 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 594/1188] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:16:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:16:47.286325 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc560c-c592-4f2e-ac53-01fcda59f2f7 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:47.399258 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc560c-c592-4f2e-ac53-01fcda59f2f7 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:47.429994 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:47.439230 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab05ce9-56ad-4b23-ae4e-4b5f309c3b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:47.440287 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab05ce9-56ad-4b23-ae4e-4b5f309c3b19 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:47.440748 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab05ce9-56ad-4b23-ae4e-4b5f309c3b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:47.441181 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab05ce9-56ad-4b23-ae4e-4b5f309c3b19 None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:47.442267 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab05ce9-56ad-4b23-ae4e-4b5f309c3b19 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:16:47.442773 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab05ce9-56ad-4b23-ae4e-4b5f309c3b19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:47.445670 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab05ce9-56ad-4b23-ae4e-4b5f309c3b19 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:47.458996 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 595/1189] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:16:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:47.465053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69447132-cda1-483b-aa37-1fe605c154e8 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:47.495505 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:47.579930 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69447132-cda1-483b-aa37-1fe605c154e8 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:47.619232 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7cf95-e3b1-478b-ba98-f5a31c774af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:47.619847 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7cf95-e3b1-478b-ba98-f5a31c774af2 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:47.620199 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7cf95-e3b1-478b-ba98-f5a31c774af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:47.620411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7cf95-e3b1-478b-ba98-f5a31c774af2 None admin] PATH_INFO: `/v3/users/695d62082a5b4d5486e74fb58cb0e05f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:47.621647 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7cf95-e3b1-478b-ba98-f5a31c774af2 None admin] RBAC: Authorizing `identity:get_user(user_id=695d62082a5b4d5486e74fb58cb0e05f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:47.623072 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7cf95-e3b1-478b-ba98-f5a31c774af2 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:47.624709 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 595/1190] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:16:47 2022] GET /identity/v3/users/695d62082a5b4d5486e74fb58cb0e05f => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:47.630795 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab05ce9-56ad-4b23-ae4e-4b5f309c3b19 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:47.741098 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab05ce9-56ad-4b23-ae4e-4b5f309c3b19 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:47.775697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5bcb6-fc01-4220-8e3a-e6fa911a87d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:47.776343 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5bcb6-fc01-4220-8e3a-e6fa911a87d4 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:47.776615 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5bcb6-fc01-4220-8e3a-e6fa911a87d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:47.776877 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5bcb6-fc01-4220-8e3a-e6fa911a87d4 None admin] PATH_INFO: `/v3/projects/da9318cc9af34bb2a97df33186ea8977` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:47.778092 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd5bcb6-fc01-4220-8e3a-e6fa911a87d4 None admin] RBAC: Authorizing `identity:get_project(project_id=da9318cc9af34bb2a97df33186ea8977)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:47.779521 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd5bcb6-fc01-4220-8e3a-e6fa911a87d4 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:47.781171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 596/1191] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:16:47 2022] GET /identity/v3/projects/da9318cc9af34bb2a97df33186ea8977 => generated 361 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:47.787736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7cf95-e3b1-478b-ba98-f5a31c774af2 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:47.887411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:47.894557 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7cf95-e3b1-478b-ba98-f5a31c774af2 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:47.932190 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be03ea3-b635-43e5-a058-e20818f5a81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:47.932834 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be03ea3-b635-43e5-a058-e20818f5a81a None admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:47.933104 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be03ea3-b635-43e5-a058-e20818f5a81a None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:47.933373 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be03ea3-b635-43e5-a058-e20818f5a81a None admin] PATH_INFO: `/v3/projects/da9318cc9af34bb2a97df33186ea8977/users/695d62082a5b4d5486e74fb58cb0e05f/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:47.935328 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be03ea3-b635-43e5-a058-e20818f5a81a None admin] RBAC: Authorizing `identity:create_grant(project_id=da9318cc9af34bb2a97df33186ea8977, user_id=695d62082a5b4d5486e74fb58cb0e05f, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:47.936769 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be03ea3-b635-43e5-a058-e20818f5a81a None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:47.953335 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 596/1192] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Aug 23 04:16:47 2022] PUT /identity/v3/projects/da9318cc9af34bb2a97df33186ea8977/users/695d62082a5b4d5486e74fb58cb0e05f/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:16:49.851526 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bb3f0-570d-4d72-b700-8ab4a8e7ab23 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:49.852096 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bb3f0-570d-4d72-b700-8ab4a8e7ab23 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:49.852096 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bb3f0-570d-4d72-b700-8ab4a8e7ab23 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:49.853383 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 597/1193] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:16:49.859887 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdae0e1-124d-4fcf-ba8f-1aa239a085ac None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:49.860698 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdae0e1-124d-4fcf-ba8f-1aa239a085ac None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:49.860954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdae0e1-124d-4fcf-ba8f-1aa239a085ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:49.882275 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-9cdae0e1-124d-4fcf-ba8f-1aa239a085ac None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:16:49.883460 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-9cdae0e1-124d-4fcf-ba8f-1aa239a085ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:16:49.956912 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 597/1194] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:16:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:16:50.221914 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8bb3f0-570d-4d72-b700-8ab4a8e7ab23 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:50.290485 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:50.340801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8bb3f0-570d-4d72-b700-8ab4a8e7ab23 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:50.376633 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f1b6e0-f364-4ea7-a4e8-72074226fe6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:50.377246 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f1b6e0-f364-4ea7-a4e8-72074226fe6d None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:50.377517 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f1b6e0-f364-4ea7-a4e8-72074226fe6d None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:50.377782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f1b6e0-f364-4ea7-a4e8-72074226fe6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:50.390074 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f1b6e0-f364-4ea7-a4e8-72074226fe6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:16:50.394635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-13f1b6e0-f364-4ea7-a4e8-72074226fe6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:16:50.394635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:16:50.394635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:16:50.394635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:16:50.394635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:16:50.394635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:16:50.394635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:16:50.394635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:16:50.394635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:16:50.394635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:16:50.394635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:16:50.394635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:16:50.394635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:16:50.394635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:16:50.394635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:16:50.394635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:16:50.395680 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:16:50.395680 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:16:50.395680 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:16:50.395680 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:16:50.395680 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:16:50.395680 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:16:50.395680 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:16:50.395680 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:16:50.395680 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:16:50.395680 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:16:50.395680 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:16:50.395680 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:16:50.395680 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:16:50.395680 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:16:50.395680 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:16:50.395680 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:16:50.397050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:16:50.397050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:16:50.397050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:16:50.397050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:16:50.397050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:16:50.397050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:16:50.397050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:16:50.397050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:16:50.397050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:16:50.397050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:16:50.397050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:16:50.397050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:16:50.397050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 598/1195] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:16:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:16:50.402102 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdae0e1-124d-4fcf-ba8f-1aa239a085ac None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:50.498033 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:50.515946 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdae0e1-124d-4fcf-ba8f-1aa239a085ac None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:50.551608 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48060b87-c36e-47e5-9e47-50e1b2f97e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:50.552237 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48060b87-c36e-47e5-9e47-50e1b2f97e40 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:50.552512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48060b87-c36e-47e5-9e47-50e1b2f97e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:50.553091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48060b87-c36e-47e5-9e47-50e1b2f97e40 None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:50.553650 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48060b87-c36e-47e5-9e47-50e1b2f97e40 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:16:50.553948 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48060b87-c36e-47e5-9e47-50e1b2f97e40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:50.555301 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48060b87-c36e-47e5-9e47-50e1b2f97e40 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:50.563479 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 598/1196] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:16:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:50.573162 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f1b6e0-f364-4ea7-a4e8-72074226fe6d None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:50.685332 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f1b6e0-f364-4ea7-a4e8-72074226fe6d None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:50.715165 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:50.737064 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab0657-50dd-45fb-9ff7-96a2573cf8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:50.738069 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab0657-50dd-45fb-9ff7-96a2573cf8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:50.738566 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab0657-50dd-45fb-9ff7-96a2573cf8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:50.739023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab0657-50dd-45fb-9ff7-96a2573cf8a7 None admin] PATH_INFO: `/v3/users/695d62082a5b4d5486e74fb58cb0e05f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:50.741041 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab0657-50dd-45fb-9ff7-96a2573cf8a7 None admin] RBAC: Authorizing `identity:get_user(user_id=695d62082a5b4d5486e74fb58cb0e05f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:50.743401 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab0657-50dd-45fb-9ff7-96a2573cf8a7 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:50.746039 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 599/1197] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Aug 23 04:16:50 2022] GET /identity/v3/users/695d62082a5b4d5486e74fb58cb0e05f => generated 311 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:16:50.755430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48060b87-c36e-47e5-9e47-50e1b2f97e40 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:50.860958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48060b87-c36e-47e5-9e47-50e1b2f97e40 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:50.894636 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1c94f-ab70-4328-a91f-216c039bfe6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:50.895230 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1c94f-ab70-4328-a91f-216c039bfe6e None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:50.895504 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1c94f-ab70-4328-a91f-216c039bfe6e None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:50.895770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1c94f-ab70-4328-a91f-216c039bfe6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:50.919349 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:50.970812 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a1c94f-ab70-4328-a91f-216c039bfe6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:50.972289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a1c94f-ab70-4328-a91f-216c039bfe6e None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:51.078298 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 599/1198] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:16:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 04:16:51.087616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab0657-50dd-45fb-9ff7-96a2573cf8a7 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:51.223171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab0657-50dd-45fb-9ff7-96a2573cf8a7 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:51.257943 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a61060-51f9-4dcf-97bf-8c23e21c50bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:51.258556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a61060-51f9-4dcf-97bf-8c23e21c50bb None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:51.258827 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a61060-51f9-4dcf-97bf-8c23e21c50bb None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:51.259091 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a61060-51f9-4dcf-97bf-8c23e21c50bb None admin] PATH_INFO: `/v3/projects/da9318cc9af34bb2a97df33186ea8977` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:51.260256 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a61060-51f9-4dcf-97bf-8c23e21c50bb None admin] RBAC: Authorizing `identity:get_project(project_id=da9318cc9af34bb2a97df33186ea8977)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:51.261682 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a61060-51f9-4dcf-97bf-8c23e21c50bb None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:51.263280 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 600/1199] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Aug 23 04:16:51 2022] GET /identity/v3/projects/da9318cc9af34bb2a97df33186ea8977 => generated 361 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:16:51.269624 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1c94f-ab70-4328-a91f-216c039bfe6e None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:16:51.322223 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:16:51.387175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1c94f-ab70-4328-a91f-216c039bfe6e None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:16:51.427031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8ad98-8560-4da7-9750-84b4736e051e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:16:51.427674 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8ad98-8560-4da7-9750-84b4736e051e None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:16:51.427949 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8ad98-8560-4da7-9750-84b4736e051e None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:16:51.428368 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8ad98-8560-4da7-9750-84b4736e051e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:16:51.429091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d8ad98-8560-4da7-9750-84b4736e051e None admin] RBAC: Adding query filter params (role.id=52b31061b72c45ccb76dd26bf31f8d3f, scope.project.id=da9318cc9af34bb2a97df33186ea8977, user.id=695d62082a5b4d5486e74fb58cb0e05f) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:16:51.429419 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d8ad98-8560-4da7-9750-84b4736e051e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:16:51.430843 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d8ad98-8560-4da7-9750-84b4736e051e None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:16:51.436631 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 600/1200] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:16:51 2022] GET /identity/v3/role_assignments?user.id=695d62082a5b4d5486e74fb58cb0e05f&scope.project.id=da9318cc9af34bb2a97df33186ea8977&role.id=52b31061b72c45ccb76dd26bf31f8d3f => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:17:01.252551 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=72 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:17:01.434079 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=44 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:17:47.054926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967cde54-fa63-467d-b269-6b30c3713f71 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:47.054926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967cde54-fa63-467d-b269-6b30c3713f71 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:47.054926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967cde54-fa63-467d-b269-6b30c3713f71 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:47.054926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 601/1201] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:17:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:17:47.061459 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b7602-1ad2-43e0-bfb8-b784aa3d5dd5 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:47.061459 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b7602-1ad2-43e0-bfb8-b784aa3d5dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:47.062304 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b7602-1ad2-43e0-bfb8-b784aa3d5dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:47.086958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-4d8b7602-1ad2-43e0-bfb8-b784aa3d5dd5 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:17:47.088652 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-4d8b7602-1ad2-43e0-bfb8-b784aa3d5dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:17:47.176783 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 601/1202] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:17:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:17:47.472521 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967cde54-fa63-467d-b269-6b30c3713f71 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:17:47.611384 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967cde54-fa63-467d-b269-6b30c3713f71 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:17:47.656162 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea1a43-d591-4675-87e3-4923217ab0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:17:47.656808 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea1a43-d591-4675-87e3-4923217ab0f1 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:47.657090 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea1a43-d591-4675-87e3-4923217ab0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:47.657387 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea1a43-d591-4675-87e3-4923217ab0f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:47.673127 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea1a43-d591-4675-87e3-4923217ab0f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:17:47.674654 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea1a43-d591-4675-87e3-4923217ab0f1 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:17:47.694505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-2bea1a43-d591-4675-87e3-4923217ab0f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:17:47.694505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:17:47.694505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:17:47.694505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:17:47.694505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:17:47.694505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:17:47.694505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:17:47.694505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:17:47.694505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:17:47.694505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:17:47.694505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:17:47.694505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:17:47.694505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:17:47.694505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:17:47.694505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:17:47.694505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:17:47.695776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:17:47.695776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:17:47.695776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:17:47.695776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:17:47.695776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:17:47.695776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:17:47.695776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:17:47.695776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:17:47.695776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:17:47.695776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:17:47.695776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:17:47.695776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:17:47.695776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:17:47.695776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:17:47.695776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:17:47.695776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:17:47.695776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:17:47.697268 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:17:47.697268 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:17:47.697268 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:17:47.699087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 602/1203] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:17:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:17:47.707450 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b7602-1ad2-43e0-bfb8-b784aa3d5dd5 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:17:47.869530 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b7602-1ad2-43e0-bfb8-b784aa3d5dd5 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:17:47.911338 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:17:47.917720 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb410a-c188-4d8e-abdd-e4bc1872309f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:17:47.918696 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb410a-c188-4d8e-abdd-e4bc1872309f None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:47.919151 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb410a-c188-4d8e-abdd-e4bc1872309f None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:47.919792 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb410a-c188-4d8e-abdd-e4bc1872309f None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:47.920492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb410a-c188-4d8e-abdd-e4bc1872309f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:17:47.920873 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb410a-c188-4d8e-abdd-e4bc1872309f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:17:47.922339 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb410a-c188-4d8e-abdd-e4bc1872309f None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:17:47.935420 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 602/1204] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:17:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:17:47.941494 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea1a43-d591-4675-87e3-4923217ab0f1 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:17:48.006493 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:17:48.066103 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea1a43-d591-4675-87e3-4923217ab0f1 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:17:48.109715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645fa81-0fab-4bd1-90c1-754e6e8527e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:17:48.110496 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645fa81-0fab-4bd1-90c1-754e6e8527e8 None admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:48.110896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645fa81-0fab-4bd1-90c1-754e6e8527e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:48.111124 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645fa81-0fab-4bd1-90c1-754e6e8527e8 None admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:48.111846 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f645fa81-0fab-4bd1-90c1-754e6e8527e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:17:48.113934 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f645fa81-0fab-4bd1-90c1-754e6e8527e8 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:17:48.154170 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 603/1205] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Aug 23 04:17:47 2022] POST /identity/v3/users => generated 251 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:17:50.333052 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a26f8-61ee-42f7-8912-7c18fd51a11c None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:50.333698 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a26f8-61ee-42f7-8912-7c18fd51a11c None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:50.333698 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a26f8-61ee-42f7-8912-7c18fd51a11c None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:50.335311 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 603/1206] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:17:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:17:50.341958 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94ee2b-28d2-45ee-93c0-c79ebec743b8 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:50.342404 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94ee2b-28d2-45ee-93c0-c79ebec743b8 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:50.342497 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94ee2b-28d2-45ee-93c0-c79ebec743b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:50.362902 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-5d94ee2b-28d2-45ee-93c0-c79ebec743b8 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:17:50.364238 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-5d94ee2b-28d2-45ee-93c0-c79ebec743b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:17:50.443303 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 604/1207] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:17:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:17:50.731310 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a26f8-61ee-42f7-8912-7c18fd51a11c None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:17:50.856799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a26f8-61ee-42f7-8912-7c18fd51a11c None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:17:50.896766 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34006122-2fb1-4613-8693-f54d777e2853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:17:50.897405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34006122-2fb1-4613-8693-f54d777e2853 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:50.897673 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34006122-2fb1-4613-8693-f54d777e2853 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:50.897945 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34006122-2fb1-4613-8693-f54d777e2853 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:50.910637 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34006122-2fb1-4613-8693-f54d777e2853 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:17:50.916232 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-34006122-2fb1-4613-8693-f54d777e2853 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:17:50.916232 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:17:50.916232 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:17:50.916232 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:17:50.916232 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:17:50.916232 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:17:50.916232 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:17:50.916232 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:17:50.916232 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:17:50.916232 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:17:50.916232 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:17:50.916232 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:17:50.916232 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:17:50.916232 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:17:50.916232 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:17:50.916232 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:17:50.917443 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:17:50.917443 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:17:50.917443 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:17:50.917443 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:17:50.917443 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:17:50.917443 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:17:50.917443 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:17:50.917443 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:17:50.917443 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:17:50.917443 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:17:50.917443 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:17:50.917443 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:17:50.917443 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:17:50.917443 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:17:50.917443 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:17:50.917443 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:17:50.918457 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:17:50.918457 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:17:50.918457 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:17:50.918457 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:17:50.918457 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:17:50.918457 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:17:50.918457 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:17:50.918457 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:17:50.918457 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:17:50.918457 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:17:50.918457 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:17:50.918457 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:17:50.923086 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 604/1208] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Aug 23 04:17:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:17:50.926176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94ee2b-28d2-45ee-93c0-c79ebec743b8 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:17:50.973592 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:17:51.047961 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94ee2b-28d2-45ee-93c0-c79ebec743b8 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:17:51.092834 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c10de-10e6-457b-9c6a-a183c3d86de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:17:51.093524 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c10de-10e6-457b-9c6a-a183c3d86de3 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:51.093909 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c10de-10e6-457b-9c6a-a183c3d86de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:51.094258 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c10de-10e6-457b-9c6a-a183c3d86de3 None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:51.095007 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1c10de-10e6-457b-9c6a-a183c3d86de3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:17:51.095315 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1c10de-10e6-457b-9c6a-a183c3d86de3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:17:51.096827 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1c10de-10e6-457b-9c6a-a183c3d86de3 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:17:51.115565 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 605/1209] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Aug 23 04:17:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:17:51.117117 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34006122-2fb1-4613-8693-f54d777e2853 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:17:51.186010 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:17:51.242809 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34006122-2fb1-4613-8693-f54d777e2853 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:17:51.286405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776333db-ca74-4a52-ac00-f1277a6d9c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:17:51.287068 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776333db-ca74-4a52-ac00-f1277a6d9c04 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:51.287498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776333db-ca74-4a52-ac00-f1277a6d9c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:51.287806 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776333db-ca74-4a52-ac00-f1277a6d9c04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:51.301024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776333db-ca74-4a52-ac00-f1277a6d9c04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:17:51.302642 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776333db-ca74-4a52-ac00-f1277a6d9c04 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:17:51.319747 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-776333db-ca74-4a52-ac00-f1277a6d9c04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:17:51.319747 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:17:51.319747 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:17:51.319747 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:17:51.319747 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:17:51.319747 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:17:51.319747 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:17:51.319747 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:17:51.319747 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:17:51.319747 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:17:51.319747 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:17:51.319747 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:17:51.319747 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:17:51.319747 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:17:51.319747 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:17:51.319747 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:17:51.320832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:17:51.320832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:17:51.320832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:17:51.320832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:17:51.320832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:17:51.320832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:17:51.320832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:17:51.320832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:17:51.320832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:17:51.320832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:17:51.320832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:17:51.320832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:17:51.320832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:17:51.320832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:17:51.320832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:17:51.320832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:17:51.320832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:17:51.321760 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:17:51.321760 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:17:51.321760 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:17:51.321760 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 605/1210] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:17:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:17:51.326966 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c10de-10e6-457b-9c6a-a183c3d86de3 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:17:51.442985 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:17:51.447225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c10de-10e6-457b-9c6a-a183c3d86de3 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:17:51.504802 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae852d1e-ca64-4788-af9e-fd4ab1d85912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:17:51.505422 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae852d1e-ca64-4788-af9e-fd4ab1d85912 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:51.505702 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae852d1e-ca64-4788-af9e-fd4ab1d85912 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:51.505984 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae852d1e-ca64-4788-af9e-fd4ab1d85912 None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:51.506695 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae852d1e-ca64-4788-af9e-fd4ab1d85912 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:17:51.507086 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae852d1e-ca64-4788-af9e-fd4ab1d85912 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:17:51.508629 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae852d1e-ca64-4788-af9e-fd4ab1d85912 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:17:51.523589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 606/1211] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:17:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:17:51.532032 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776333db-ca74-4a52-ac00-f1277a6d9c04 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:17:51.648046 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776333db-ca74-4a52-ac00-f1277a6d9c04 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:17:51.648046 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:17:51.684404 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeab1e6-328b-4909-be77-61757d66af1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:17:51.685113 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeab1e6-328b-4909-be77-61757d66af1b None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:51.685432 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeab1e6-328b-4909-be77-61757d66af1b None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:51.685776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeab1e6-328b-4909-be77-61757d66af1b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:51.694157 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeab1e6-328b-4909-be77-61757d66af1b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:17:51.695877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeab1e6-328b-4909-be77-61757d66af1b None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:17:51.708468 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-bbeab1e6-328b-4909-be77-61757d66af1b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 04:17:51.708468 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:17:51.708468 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:17:51.708468 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:17:51.708468 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:17:51.708468 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:17:51.708468 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:17:51.708468 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:17:51.708468 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:17:51.708468 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:17:51.708468 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:17:51.708468 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:17:51.708468 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:17:51.708468 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:17:51.708468 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:17:51.708468 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:17:51.709592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:17:51.709592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:17:51.709592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:17:51.709592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:17:51.709592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:17:51.709592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:17:51.709592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:17:51.709592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:17:51.709592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:17:51.709592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:17:51.709592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:17:51.709592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:17:51.709592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:17:51.709592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:17:51.709592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:17:51.709592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:17:51.709592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:17:51.709592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:17:51.710549 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:17:51.710549 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:17:51.710549 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:17:51.710549 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:17:51.710549 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:17:51.710549 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:17:51.710549 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:17:51.710549 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:17:51.710549 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 04:17:51.710549 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:17:51.710549 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 606/1212] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:17:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:17:51.715889 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae852d1e-ca64-4788-af9e-fd4ab1d85912 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:17:51.836110 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae852d1e-ca64-4788-af9e-fd4ab1d85912 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:17:51.885762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53cad04-9545-40d5-b589-48f548862480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:17:51.886773 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53cad04-9545-40d5-b589-48f548862480 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:51.887067 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53cad04-9545-40d5-b589-48f548862480 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:51.887332 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53cad04-9545-40d5-b589-48f548862480 None admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:51.888303 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53cad04-9545-40d5-b589-48f548862480 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:17:51.888638 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53cad04-9545-40d5-b589-48f548862480 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:17:51.890221 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53cad04-9545-40d5-b589-48f548862480 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:17:51.911665 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 607/1213] 10.222.0.18 () {56 vars in 1143 bytes} [Tue Aug 23 04:17:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:17:51.917367 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeab1e6-328b-4909-be77-61757d66af1b None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:17:52.035431 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeab1e6-328b-4909-be77-61757d66af1b None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:17:52.079401 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d28e23-11d0-45f8-8cc2-761abd9b707e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:17:52.080189 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d28e23-11d0-45f8-8cc2-761abd9b707e None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:52.080500 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d28e23-11d0-45f8-8cc2-761abd9b707e None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:52.082976 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d28e23-11d0-45f8-8cc2-761abd9b707e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:52.118046 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:17:52.169418 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d28e23-11d0-45f8-8cc2-761abd9b707e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:17:52.171474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d28e23-11d0-45f8-8cc2-761abd9b707e None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:17:52.313191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 607/1214] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:17:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 04:17:52.319657 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53cad04-9545-40d5-b589-48f548862480 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:17:52.342114 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:17:52.434085 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53cad04-9545-40d5-b589-48f548862480 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:17:52.479334 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f8e3d-0cec-411b-b8dd-6c8be0efad69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:17:52.480014 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f8e3d-0cec-411b-b8dd-6c8be0efad69 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:52.480287 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f8e3d-0cec-411b-b8dd-6c8be0efad69 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:52.480573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f8e3d-0cec-411b-b8dd-6c8be0efad69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:52.494459 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f8e3d-0cec-411b-b8dd-6c8be0efad69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:17:52.495981 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f8e3d-0cec-411b-b8dd-6c8be0efad69 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:17:52.512693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-712f8e3d-0cec-411b-b8dd-6c8be0efad69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:17:52.512693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:17:52.512693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:17:52.512693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:17:52.512693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:17:52.512693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:17:52.512693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:17:52.512693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:17:52.512693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:17:52.512693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:17:52.512693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:17:52.512693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:17:52.512693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:17:52.512693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:17:52.512693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:17:52.512693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:17:52.513822 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:17:52.513822 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:17:52.513822 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:17:52.513822 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:17:52.513822 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:17:52.513822 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:17:52.513822 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:17:52.513822 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:17:52.513822 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:17:52.513822 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:17:52.513822 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:17:52.513822 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:17:52.513822 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:17:52.513822 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:17:52.513822 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:17:52.513822 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:17:52.513822 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:17:52.514699 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:17:52.514699 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:17:52.514699 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:17:52.514699 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 608/1215] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:17:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:17:52.520873 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d28e23-11d0-45f8-8cc2-761abd9b707e None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:17:52.590919 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:17:52.633933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d28e23-11d0-45f8-8cc2-761abd9b707e None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:17:52.672991 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f277f-9c97-4133-a364-ec387bc2072e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:17:52.674724 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f277f-9c97-4133-a364-ec387bc2072e None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:52.674724 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f277f-9c97-4133-a364-ec387bc2072e None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:52.674724 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f277f-9c97-4133-a364-ec387bc2072e None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:52.674965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8f277f-9c97-4133-a364-ec387bc2072e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:17:52.675175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8f277f-9c97-4133-a364-ec387bc2072e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:17:52.676764 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8f277f-9c97-4133-a364-ec387bc2072e None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:17:52.690055 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 608/1216] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:17:52.696470 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f8e3d-0cec-411b-b8dd-6c8be0efad69 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:17:52.774058 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:17:52.808220 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f8e3d-0cec-411b-b8dd-6c8be0efad69 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:17:52.845839 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5bfc4c-feb4-42a7-b4c1-0dac9288842f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:17:52.846563 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5bfc4c-feb4-42a7-b4c1-0dac9288842f None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:52.846850 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5bfc4c-feb4-42a7-b4c1-0dac9288842f None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:52.847090 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5bfc4c-feb4-42a7-b4c1-0dac9288842f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:52.854891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5bfc4c-feb4-42a7-b4c1-0dac9288842f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:17:52.856416 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5bfc4c-feb4-42a7-b4c1-0dac9288842f None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:17:52.869370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-ce5bfc4c-feb4-42a7-b4c1-0dac9288842f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:17:52.869370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:17:52.869370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:17:52.869370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:17:52.869370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:17:52.869370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:17:52.869370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:17:52.869370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:17:52.869370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:17:52.869370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:17:52.869370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:17:52.869370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:17:52.869370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:17:52.869370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:17:52.869370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:17:52.870552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:17:52.870552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:17:52.870552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:17:52.870552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:17:52.870552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:17:52.870552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:17:52.870552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:17:52.870552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:17:52.870552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:17:52.870552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:17:52.870552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:17:52.870552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:17:52.870552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:17:52.870552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:17:52.870552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:17:52.870552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:17:52.870552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:17:52.871549 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:17:52.871549 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:17:52.871549 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:17:52.871549 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:17:52.871549 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:17:52.871549 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:17:52.871549 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:17:52.871549 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:17:52.871549 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 609/1217] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:17:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:17:52.878020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f277f-9c97-4133-a364-ec387bc2072e None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:17:52.994540 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f277f-9c97-4133-a364-ec387bc2072e None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:17:53.017458 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:17:53.031597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6ca0d-2126-4b8d-848d-a7e340942124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:17:53.032281 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6ca0d-2126-4b8d-848d-a7e340942124 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:53.032561 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6ca0d-2126-4b8d-848d-a7e340942124 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:53.032887 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6ca0d-2126-4b8d-848d-a7e340942124 None admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:53.033589 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f6ca0d-2126-4b8d-848d-a7e340942124 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:17:53.033897 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f6ca0d-2126-4b8d-848d-a7e340942124 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:17:53.035464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f6ca0d-2126-4b8d-848d-a7e340942124 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:17:53.053974 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 609/1218] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:17:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:17:53.055964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5bfc4c-feb4-42a7-b4c1-0dac9288842f None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:17:53.189631 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5bfc4c-feb4-42a7-b4c1-0dac9288842f None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:17:53.228634 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:17:53.236340 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44701734-27d2-4456-94ff-27dc3265b0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:17:53.236959 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44701734-27d2-4456-94ff-27dc3265b0bf None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:53.237237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44701734-27d2-4456-94ff-27dc3265b0bf None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:53.237508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44701734-27d2-4456-94ff-27dc3265b0bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:53.238247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44701734-27d2-4456-94ff-27dc3265b0bf None admin] RBAC: Adding query filter params (role.id=bdf71ef6ab10430e8c356cd4d952ff5c, scope.project.id=9f4373dfb5f9425eb73669e24d18a08f, user.id=dd503fe3cf6d428e962c14ab864ee608) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:17:53.238545 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44701734-27d2-4456-94ff-27dc3265b0bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:17:53.240040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44701734-27d2-4456-94ff-27dc3265b0bf None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:17:53.246218 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 610/1219] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:17:53 2022] GET /identity/v3/role_assignments?user.id=dd503fe3cf6d428e962c14ab864ee608&scope.project.id=9f4373dfb5f9425eb73669e24d18a08f&role.id=bdf71ef6ab10430e8c356cd4d952ff5c => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:17:55.292891 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d486ac-be2d-416b-9da1-9ff5cac32ecb None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:55.293479 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d486ac-be2d-416b-9da1-9ff5cac32ecb None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:55.293479 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d486ac-be2d-416b-9da1-9ff5cac32ecb None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:55.294789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 610/1220] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:17:55.301778 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de467c4-1e27-4777-9407-01974bdccf94 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:55.302225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de467c4-1e27-4777-9407-01974bdccf94 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:55.302318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de467c4-1e27-4777-9407-01974bdccf94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:55.322395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-8de467c4-1e27-4777-9407-01974bdccf94 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:17:55.323667 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-8de467c4-1e27-4777-9407-01974bdccf94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:17:55.398515 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 611/1221] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:17:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:17:55.670955 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d486ac-be2d-416b-9da1-9ff5cac32ecb None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:17:55.785406 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d486ac-be2d-416b-9da1-9ff5cac32ecb None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:17:55.830045 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ae938-d820-4b83-9785-1ebb2a088c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:17:55.830746 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ae938-d820-4b83-9785-1ebb2a088c9e None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:55.831057 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ae938-d820-4b83-9785-1ebb2a088c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:55.831364 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ae938-d820-4b83-9785-1ebb2a088c9e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:55.843199 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ae938-d820-4b83-9785-1ebb2a088c9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:17:55.848247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-b49ae938-d820-4b83-9785-1ebb2a088c9e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:17:55.848247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:17:55.848247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:17:55.848247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:17:55.848247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:17:55.848247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:17:55.848247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:17:55.848247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:17:55.848247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:17:55.848247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:17:55.848247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:17:55.848247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:17:55.848247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:17:55.848247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:17:55.848247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:17:55.848247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:17:55.849662 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:17:55.849662 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:17:55.849662 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:17:55.849662 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:17:55.849662 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:17:55.849662 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:17:55.849662 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:17:55.849662 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:17:55.849662 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:17:55.849662 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:17:55.849662 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:17:55.849662 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:17:55.849662 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:17:55.849662 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:17:55.849662 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:17:55.849662 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:17:55.850834 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:17:55.850834 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:17:55.850834 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:17:55.850834 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:17:55.850834 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:17:55.850834 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:17:55.850834 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:17:55.850834 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:17:55.850834 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:17:55.850834 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:17:55.850834 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:17:55.850834 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:17:55.855514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 611/1222] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Aug 23 04:17:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:17:55.857923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de467c4-1e27-4777-9407-01974bdccf94 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:17:55.973865 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de467c4-1e27-4777-9407-01974bdccf94 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:17:56.002184 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:17:56.013833 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470f6c9-2ffb-4745-8232-0c31c1d033c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:17:56.014485 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470f6c9-2ffb-4745-8232-0c31c1d033c0 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:56.014767 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470f6c9-2ffb-4745-8232-0c31c1d033c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:56.015100 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470f6c9-2ffb-4745-8232-0c31c1d033c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:56.015801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470f6c9-2ffb-4745-8232-0c31c1d033c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:17:56.016152 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470f6c9-2ffb-4745-8232-0c31c1d033c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:17:56.017681 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470f6c9-2ffb-4745-8232-0c31c1d033c0 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:17:56.027281 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 612/1223] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Aug 23 04:17:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:17:56.034022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ae938-d820-4b83-9785-1ebb2a088c9e None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:17:56.081364 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:17:56.145046 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ae938-d820-4b83-9785-1ebb2a088c9e None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:17:56.183861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45fc8a-3794-4809-bcba-2b901c313495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:17:56.184518 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45fc8a-3794-4809-bcba-2b901c313495 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:56.184797 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45fc8a-3794-4809-bcba-2b901c313495 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:56.185068 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45fc8a-3794-4809-bcba-2b901c313495 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:56.195827 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd45fc8a-3794-4809-bcba-2b901c313495 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:17:56.197459 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd45fc8a-3794-4809-bcba-2b901c313495 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:17:56.214007 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-bd45fc8a-3794-4809-bcba-2b901c313495 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:17:56.214007 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:17:56.214007 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:17:56.214007 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:17:56.214007 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:17:56.214007 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:17:56.214007 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:17:56.214007 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:17:56.214007 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:17:56.214007 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:17:56.214007 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:17:56.214007 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:17:56.214007 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:17:56.214007 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:17:56.214007 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:17:56.214007 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:17:56.215002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:17:56.215002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:17:56.215002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:17:56.215002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:17:56.215002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:17:56.215002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:17:56.215002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:17:56.215002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:17:56.215002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:17:56.215002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:17:56.215002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:17:56.215002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:17:56.215002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:17:56.215002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:17:56.215002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:17:56.215002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:17:56.215002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:17:56.215873 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:17:56.215873 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:17:56.215873 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:17:56.215873 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 612/1224] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:17:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:17:56.221849 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470f6c9-2ffb-4745-8232-0c31c1d033c0 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:17:56.330565 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470f6c9-2ffb-4745-8232-0c31c1d033c0 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:17:56.367109 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922cdc44-333c-4bc8-ad9e-803c43fc7fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:17:56.367730 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cdc44-333c-4bc8-ad9e-803c43fc7fab None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:56.368033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cdc44-333c-4bc8-ad9e-803c43fc7fab None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:56.368299 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cdc44-333c-4bc8-ad9e-803c43fc7fab None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:56.368945 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922cdc44-333c-4bc8-ad9e-803c43fc7fab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:17:56.369243 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922cdc44-333c-4bc8-ad9e-803c43fc7fab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:17:56.370617 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922cdc44-333c-4bc8-ad9e-803c43fc7fab None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:17:56.383147 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 613/1225] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:17:56.389414 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45fc8a-3794-4809-bcba-2b901c313495 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:17:56.496381 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:17:56.503220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45fc8a-3794-4809-bcba-2b901c313495 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:17:56.541225 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c821572-fb54-4e98-bff4-7c3d11bc9aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:17:56.541911 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c821572-fb54-4e98-bff4-7c3d11bc9aa2 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:56.542202 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c821572-fb54-4e98-bff4-7c3d11bc9aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:56.542469 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c821572-fb54-4e98-bff4-7c3d11bc9aa2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:56.552465 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c821572-fb54-4e98-bff4-7c3d11bc9aa2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:17:56.554356 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c821572-fb54-4e98-bff4-7c3d11bc9aa2 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:17:56.567191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-4c821572-fb54-4e98-bff4-7c3d11bc9aa2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 04:17:56.567191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:17:56.567191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:17:56.567191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:17:56.567191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:17:56.567191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:17:56.567191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:17:56.567191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:17:56.567191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:17:56.567191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:17:56.567191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:17:56.567191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:17:56.567191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:17:56.567191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:17:56.567191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:17:56.567191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:17:56.568278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:17:56.568278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:17:56.568278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:17:56.568278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:17:56.568278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:17:56.568278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:17:56.568278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:17:56.568278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:17:56.568278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:17:56.568278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:17:56.568278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:17:56.568278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:17:56.568278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:17:56.568278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:17:56.568278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:17:56.568278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:17:56.568278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:17:56.568278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:17:56.569247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:17:56.569247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:17:56.569247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:17:56.569247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:17:56.569247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:17:56.569247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:17:56.569247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:17:56.569247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:17:56.569247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 04:17:56.569247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:17:56.569247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 613/1226] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:17:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:17:56.575050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922cdc44-333c-4bc8-ad9e-803c43fc7fab None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:17:56.637329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:17:56.697699 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922cdc44-333c-4bc8-ad9e-803c43fc7fab None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:17:56.735666 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e444fb-83ae-4184-89cf-2159d704abc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:17:56.736308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e444fb-83ae-4184-89cf-2159d704abc2 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:56.736613 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e444fb-83ae-4184-89cf-2159d704abc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:56.736937 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e444fb-83ae-4184-89cf-2159d704abc2 None admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:56.737687 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e444fb-83ae-4184-89cf-2159d704abc2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:17:56.737972 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e444fb-83ae-4184-89cf-2159d704abc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:17:56.739439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e444fb-83ae-4184-89cf-2159d704abc2 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:17:56.759956 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 614/1227] 10.222.0.18 () {56 vars in 1143 bytes} [Tue Aug 23 04:17:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:17:56.764677 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c821572-fb54-4e98-bff4-7c3d11bc9aa2 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:17:56.871539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c821572-fb54-4e98-bff4-7c3d11bc9aa2 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:17:56.910894 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ee3fd-466a-43dd-be0a-a79d2fff7d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:17:56.911573 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ee3fd-466a-43dd-be0a-a79d2fff7d41 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:56.911855 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ee3fd-466a-43dd-be0a-a79d2fff7d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:56.912299 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ee3fd-466a-43dd-be0a-a79d2fff7d41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:56.923806 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ee3fd-466a-43dd-be0a-a79d2fff7d41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:17:56.925400 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ee3fd-466a-43dd-be0a-a79d2fff7d41 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:17:56.940765 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-3c4ee3fd-466a-43dd-be0a-a79d2fff7d41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:17:56.940765 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:17:56.940765 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:17:56.940765 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:17:56.940765 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:17:56.940765 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:17:56.940765 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:17:56.940765 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:17:56.940765 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:17:56.940765 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:17:56.940765 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:17:56.940765 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:17:56.940765 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:17:56.940765 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:17:56.940765 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:17:56.940765 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:17:56.941828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:17:56.941828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:17:56.941828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:17:56.941828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:17:56.941828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:17:56.941828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:17:56.941828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:17:56.941828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:17:56.941828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:17:56.941828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:17:56.941828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:17:56.941828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:17:56.941828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:17:56.941828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:17:56.941828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:17:56.941828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:17:56.941828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:17:56.942745 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:17:56.942745 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:17:56.942745 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:17:56.942745 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 614/1228] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:17:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:17:56.948201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e444fb-83ae-4184-89cf-2159d704abc2 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:17:57.065934 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e444fb-83ae-4184-89cf-2159d704abc2 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:17:57.069148 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:17:57.105000 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0571fe0-8761-4e50-ae39-289fb70a3169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:17:57.105621 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0571fe0-8761-4e50-ae39-289fb70a3169 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:57.105901 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0571fe0-8761-4e50-ae39-289fb70a3169 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:57.106173 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0571fe0-8761-4e50-ae39-289fb70a3169 None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:57.106818 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0571fe0-8761-4e50-ae39-289fb70a3169 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:17:57.107124 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0571fe0-8761-4e50-ae39-289fb70a3169 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:17:57.108532 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0571fe0-8761-4e50-ae39-289fb70a3169 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:17:57.120625 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 615/1229] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:17:57.127127 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ee3fd-466a-43dd-be0a-a79d2fff7d41 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:17:57.137690 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:17:57.241527 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ee3fd-466a-43dd-be0a-a79d2fff7d41 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:17:57.278955 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710c0aa-7c4a-4cd7-8cf7-1f7b0b121736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:17:57.280145 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710c0aa-7c4a-4cd7-8cf7-1f7b0b121736 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:57.280145 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710c0aa-7c4a-4cd7-8cf7-1f7b0b121736 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:57.280145 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710c0aa-7c4a-4cd7-8cf7-1f7b0b121736 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:57.290826 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3710c0aa-7c4a-4cd7-8cf7-1f7b0b121736 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:17:57.293308 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3710c0aa-7c4a-4cd7-8cf7-1f7b0b121736 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:17:57.311250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-3710c0aa-7c4a-4cd7-8cf7-1f7b0b121736 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:17:57.311250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:17:57.311250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:17:57.311250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:17:57.311250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:17:57.311250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:17:57.311250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:17:57.311250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:17:57.311250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:17:57.311250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:17:57.311250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:17:57.311250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:17:57.311250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:17:57.311250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:17:57.311250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:17:57.312514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:17:57.312514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:17:57.312514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:17:57.312514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:17:57.312514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:17:57.312514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:17:57.312514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:17:57.312514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:17:57.312514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:17:57.312514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:17:57.312514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:17:57.312514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:17:57.312514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:17:57.312514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:17:57.312514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:17:57.312514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:17:57.312514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:17:57.313604 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:17:57.313604 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:17:57.313604 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:17:57.313604 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:17:57.313604 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:17:57.313604 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:17:57.313604 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:17:57.313604 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:17:57.313604 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 615/1230] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:17:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:17:57.319166 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0571fe0-8761-4e50-ae39-289fb70a3169 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:17:57.431231 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0571fe0-8761-4e50-ae39-289fb70a3169 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:17:57.471617 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98220472-838c-4f69-91de-00478d752136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:17:57.472300 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98220472-838c-4f69-91de-00478d752136 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:57.472582 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98220472-838c-4f69-91de-00478d752136 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:57.472860 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98220472-838c-4f69-91de-00478d752136 None admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:57.473563 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98220472-838c-4f69-91de-00478d752136 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:17:57.473882 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98220472-838c-4f69-91de-00478d752136 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:17:57.475569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98220472-838c-4f69-91de-00478d752136 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:17:57.489703 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 616/1231] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:17:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:17:57.496152 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710c0aa-7c4a-4cd7-8cf7-1f7b0b121736 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:17:57.566353 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:17:57.610637 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710c0aa-7c4a-4cd7-8cf7-1f7b0b121736 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:17:57.646545 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede2e76f-b8de-4af9-ac33-5a3be40ec8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:17:57.647286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2e76f-b8de-4af9-ac33-5a3be40ec8af None admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:57.647582 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2e76f-b8de-4af9-ac33-5a3be40ec8af None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:57.647893 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2e76f-b8de-4af9-ac33-5a3be40ec8af None admin] PATH_INFO: `/v3/projects/9f4373dfb5f9425eb73669e24d18a08f/users/dd503fe3cf6d428e962c14ab864ee608/roles/bdf71ef6ab10430e8c356cd4d952ff5c` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:57.664111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede2e76f-b8de-4af9-ac33-5a3be40ec8af None admin] RBAC: Authorizing `identity:create_grant(project_id=9f4373dfb5f9425eb73669e24d18a08f, user_id=dd503fe3cf6d428e962c14ab864ee608, role_id=bdf71ef6ab10430e8c356cd4d952ff5c)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:17:57.665684 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede2e76f-b8de-4af9-ac33-5a3be40ec8af None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:17:57.680498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 616/1232] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Aug 23 04:17:57 2022] PUT /identity/v3/projects/9f4373dfb5f9425eb73669e24d18a08f/users/dd503fe3cf6d428e962c14ab864ee608/roles/bdf71ef6ab10430e8c356cd4d952ff5c => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:17:59.660283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5a849-1fe0-4769-8224-1246ca44f5fc None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:59.660283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5a849-1fe0-4769-8224-1246ca44f5fc None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:59.660283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5a849-1fe0-4769-8224-1246ca44f5fc None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:59.661025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 617/1233] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:17:59.666733 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b557ed-a017-41c4-ae59-d4f2803c784c None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:17:59.667239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b557ed-a017-41c4-ae59-d4f2803c784c None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:17:59.667335 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b557ed-a017-41c4-ae59-d4f2803c784c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:17:59.686397 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-28b557ed-a017-41c4-ae59-d4f2803c784c None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:17:59.687656 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-28b557ed-a017-41c4-ae59-d4f2803c784c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:17:59.767694 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 617/1234] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:17:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:18:00.038791 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c5a849-1fe0-4769-8224-1246ca44f5fc None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:00.070076 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:00.154801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c5a849-1fe0-4769-8224-1246ca44f5fc None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:00.191542 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb94f4-3313-4b63-8f16-d22184ad7dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:00.192213 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb94f4-3313-4b63-8f16-d22184ad7dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:00.192485 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb94f4-3313-4b63-8f16-d22184ad7dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:00.192750 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb94f4-3313-4b63-8f16-d22184ad7dc7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:00.203361 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acb94f4-3313-4b63-8f16-d22184ad7dc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:18:00.208622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-0acb94f4-3313-4b63-8f16-d22184ad7dc7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:18:00.208622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:18:00.208622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:18:00.208622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:18:00.208622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:18:00.208622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:18:00.208622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:18:00.208622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:18:00.208622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:18:00.208622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:18:00.208622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:18:00.208622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:18:00.208622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:18:00.208622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:18:00.208622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:18:00.208622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:18:00.210716 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:18:00.210716 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:18:00.210716 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:18:00.210716 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:18:00.210716 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:18:00.210716 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:18:00.210716 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:18:00.210716 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:18:00.210716 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:18:00.210716 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:18:00.210716 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:18:00.210716 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:18:00.210716 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:18:00.210716 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:18:00.210716 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:18:00.210716 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:18:00.211999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:18:00.211999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:18:00.211999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:18:00.211999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:18:00.211999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:18:00.211999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:18:00.211999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:18:00.211999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:18:00.211999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:18:00.211999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:18:00.211999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 04:18:00.211999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:18:00.211999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 618/1235] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Aug 23 04:18:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:18:00.217636 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b557ed-a017-41c4-ae59-d4f2803c784c None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:00.271742 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:00.328997 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b557ed-a017-41c4-ae59-d4f2803c784c None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:00.365183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1cf9ff-5c00-4482-9bfd-8d7a2c1f02fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:00.365957 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cf9ff-5c00-4482-9bfd-8d7a2c1f02fd None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:00.366454 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cf9ff-5c00-4482-9bfd-8d7a2c1f02fd None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:00.366758 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cf9ff-5c00-4482-9bfd-8d7a2c1f02fd None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:00.367474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1cf9ff-5c00-4482-9bfd-8d7a2c1f02fd None admin] RBAC: Adding query filter params (name=service) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:18:00.367793 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1cf9ff-5c00-4482-9bfd-8d7a2c1f02fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:00.369228 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1cf9ff-5c00-4482-9bfd-8d7a2c1f02fd None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:00.378663 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 618/1236] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Aug 23 04:18:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:18:00.384610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb94f4-3313-4b63-8f16-d22184ad7dc7 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:00.478026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:00.493998 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb94f4-3313-4b63-8f16-d22184ad7dc7 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:00.529170 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3b2a0-908b-40fa-a27b-859e620ee53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:00.529798 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3b2a0-908b-40fa-a27b-859e620ee53a None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:00.530164 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3b2a0-908b-40fa-a27b-859e620ee53a None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:00.530390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3b2a0-908b-40fa-a27b-859e620ee53a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:00.541566 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f3b2a0-908b-40fa-a27b-859e620ee53a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:00.543070 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f3b2a0-908b-40fa-a27b-859e620ee53a None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:00.557579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-b6f3b2a0-908b-40fa-a27b-859e620ee53a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:18:00.557579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:18:00.557579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:18:00.557579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:18:00.557579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:18:00.557579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:18:00.557579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:18:00.557579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:18:00.557579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:18:00.557579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:18:00.557579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:18:00.557579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:18:00.557579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:18:00.557579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:18:00.557579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:18:00.557579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:18:00.558709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:18:00.558709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:18:00.558709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:18:00.558709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:18:00.558709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:18:00.558709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:18:00.558709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:18:00.558709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:18:00.558709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:18:00.558709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:18:00.558709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:18:00.558709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:18:00.558709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:18:00.558709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:18:00.558709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:18:00.558709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:18:00.558709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:18:00.559614 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:18:00.559614 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:18:00.559614 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:18:00.559614 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 619/1237] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:18:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:18:00.564834 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1cf9ff-5c00-4482-9bfd-8d7a2c1f02fd None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:00.674550 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1cf9ff-5c00-4482-9bfd-8d7a2c1f02fd None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:00.689125 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:00.717880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e6ba6-379a-4469-ba78-1561eff42fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:00.718539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e6ba6-379a-4469-ba78-1561eff42fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:00.718860 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e6ba6-379a-4469-ba78-1561eff42fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:00.719131 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e6ba6-379a-4469-ba78-1561eff42fc6 None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:00.719776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570e6ba6-379a-4469-ba78-1561eff42fc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:18:00.720130 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570e6ba6-379a-4469-ba78-1561eff42fc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:00.721618 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570e6ba6-379a-4469-ba78-1561eff42fc6 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:00.735120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 619/1238] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:18:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:18:00.741385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3b2a0-908b-40fa-a27b-859e620ee53a None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:00.857752 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3b2a0-908b-40fa-a27b-859e620ee53a None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:00.893572 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b04896-8e75-4970-a2b4-556847e33b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:00.894360 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b04896-8e75-4970-a2b4-556847e33b14 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:00.894685 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b04896-8e75-4970-a2b4-556847e33b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:00.895097 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b04896-8e75-4970-a2b4-556847e33b14 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:00.898799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:00.906018 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b04896-8e75-4970-a2b4-556847e33b14 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:00.908110 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b04896-8e75-4970-a2b4-556847e33b14 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:00.920558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-83b04896-8e75-4970-a2b4-556847e33b14 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 04:18:00.920558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:18:00.920558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:18:00.920558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:18:00.920558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:18:00.920558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:18:00.920558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:18:00.920558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:18:00.920558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:18:00.920558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:18:00.920558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:18:00.920558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:18:00.920558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:18:00.920558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:18:00.920558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:18:00.920558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:18:00.922019 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:18:00.922019 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:18:00.922019 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:18:00.922019 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:18:00.922019 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:18:00.922019 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:18:00.922019 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:18:00.922019 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:18:00.922019 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:18:00.922019 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:18:00.922019 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:18:00.922019 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:18:00.922019 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:18:00.922019 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:18:00.922019 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:18:00.922019 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:18:00.922019 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:18:00.922019 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:18:00.923406 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:18:00.923406 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:18:00.923406 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:18:00.923406 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:18:00.923406 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:18:00.923406 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:18:00.923406 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:18:00.923406 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:18:00.923406 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 04:18:00.923406 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:18:00.923406 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 620/1239] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:18:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:18:00.928013 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e6ba6-379a-4469-ba78-1561eff42fc6 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:01.043769 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e6ba6-379a-4469-ba78-1561eff42fc6 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:01.083359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6f9d8-88a6-4afc-af4c-b60fd3dfb868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:01.084018 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6f9d8-88a6-4afc-af4c-b60fd3dfb868 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:01.084300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6f9d8-88a6-4afc-af4c-b60fd3dfb868 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:01.084579 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6f9d8-88a6-4afc-af4c-b60fd3dfb868 None admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:01.085317 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6f9d8-88a6-4afc-af4c-b60fd3dfb868 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:18:01.085622 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6f9d8-88a6-4afc-af4c-b60fd3dfb868 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:01.087121 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6f9d8-88a6-4afc-af4c-b60fd3dfb868 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:01.106244 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 620/1240] 10.222.0.18 () {56 vars in 1143 bytes} [Tue Aug 23 04:18:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:18:01.112234 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b04896-8e75-4970-a2b4-556847e33b14 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:01.221508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b04896-8e75-4970-a2b4-556847e33b14 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:01.257441 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db352ca-d403-4e79-9997-f60ef0580907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:01.258054 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db352ca-d403-4e79-9997-f60ef0580907 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:01.258409 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db352ca-d403-4e79-9997-f60ef0580907 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:01.258701 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db352ca-d403-4e79-9997-f60ef0580907 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:01.325872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:01.333288 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db352ca-d403-4e79-9997-f60ef0580907 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:01.335542 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db352ca-d403-4e79-9997-f60ef0580907 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:01.441403 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 621/1241] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:18:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 04:18:01.448330 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6f9d8-88a6-4afc-af4c-b60fd3dfb868 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:01.482058 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:01.564607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6f9d8-88a6-4afc-af4c-b60fd3dfb868 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:01.604829 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3e704-f3c6-42a8-b1b6-106c61c9a9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:01.605531 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3e704-f3c6-42a8-b1b6-106c61c9a9b9 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:01.605852 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3e704-f3c6-42a8-b1b6-106c61c9a9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:01.606144 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3e704-f3c6-42a8-b1b6-106c61c9a9b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:01.616581 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3e704-f3c6-42a8-b1b6-106c61c9a9b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:01.618027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3e704-f3c6-42a8-b1b6-106c61c9a9b9 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:01.632958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-a3f3e704-f3c6-42a8-b1b6-106c61c9a9b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:18:01.632958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:18:01.632958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:18:01.632958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:18:01.632958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:18:01.632958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:18:01.632958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:18:01.632958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:18:01.632958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:18:01.632958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:18:01.632958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:18:01.632958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:18:01.632958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:18:01.632958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:18:01.632958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:18:01.632958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:18:01.634175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:18:01.634175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:18:01.634175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:18:01.634175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:18:01.634175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:18:01.634175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:18:01.634175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:18:01.634175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:18:01.634175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:18:01.634175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:18:01.634175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:18:01.634175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:18:01.634175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:18:01.634175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:18:01.634175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:18:01.634175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:18:01.634175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:18:01.635252 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:18:01.635252 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:18:01.635252 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:18:01.635252 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 621/1242] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:18:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:18:01.640053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db352ca-d403-4e79-9997-f60ef0580907 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:01.748855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:01.754879 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db352ca-d403-4e79-9997-f60ef0580907 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:01.793338 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476a54e-a0c8-45e6-be22-cb820ea2fae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:01.793948 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476a54e-a0c8-45e6-be22-cb820ea2fae0 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:01.794222 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476a54e-a0c8-45e6-be22-cb820ea2fae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:01.794487 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476a54e-a0c8-45e6-be22-cb820ea2fae0 None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:01.795123 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476a54e-a0c8-45e6-be22-cb820ea2fae0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:18:01.795453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476a54e-a0c8-45e6-be22-cb820ea2fae0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:01.796964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476a54e-a0c8-45e6-be22-cb820ea2fae0 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:01.809376 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 622/1243] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:18:01.815357 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3e704-f3c6-42a8-b1b6-106c61c9a9b9 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:01.904907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:01.929901 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3e704-f3c6-42a8-b1b6-106c61c9a9b9 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:01.968941 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221e6d2-3dd4-406e-a4c4-d52224ae27b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:01.969581 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221e6d2-3dd4-406e-a4c4-d52224ae27b6 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:01.969922 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221e6d2-3dd4-406e-a4c4-d52224ae27b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:01.970230 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221e6d2-3dd4-406e-a4c4-d52224ae27b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:01.978030 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e221e6d2-3dd4-406e-a4c4-d52224ae27b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:01.979530 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e221e6d2-3dd4-406e-a4c4-d52224ae27b6 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:01.991628 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-e221e6d2-3dd4-406e-a4c4-d52224ae27b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:18:01.991628 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:18:01.991628 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:18:01.991628 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:18:01.991628 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:18:01.991628 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:18:01.991628 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:18:01.991628 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:18:01.991628 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:18:01.991628 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:18:01.991628 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:18:01.991628 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:18:01.991628 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:18:01.991628 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:18:01.991628 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:18:01.992883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:18:01.992883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:18:01.992883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:18:01.992883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:18:01.992883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:18:01.992883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:18:01.992883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:18:01.992883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:18:01.992883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:18:01.992883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:18:01.992883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:18:01.992883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:18:01.992883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:18:01.992883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:18:01.992883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:18:01.992883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:18:01.992883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:18:01.994161 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:18:01.994161 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:18:01.994161 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:18:01.994161 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:18:01.994161 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:18:01.994161 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:18:01.994161 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:18:01.994161 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:18:01.994161 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 622/1244] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:18:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:18:01.999171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476a54e-a0c8-45e6-be22-cb820ea2fae0 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:02.110602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476a54e-a0c8-45e6-be22-cb820ea2fae0 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:02.148803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e54678-1e7c-458b-9b8c-7889804633d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:02.149530 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e54678-1e7c-458b-9b8c-7889804633d9 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:02.149810 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e54678-1e7c-458b-9b8c-7889804633d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:02.150151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e54678-1e7c-458b-9b8c-7889804633d9 None admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:02.150904 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e54678-1e7c-458b-9b8c-7889804633d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:18:02.151207 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e54678-1e7c-458b-9b8c-7889804633d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:02.152875 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e54678-1e7c-458b-9b8c-7889804633d9 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:02.171654 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 623/1245] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:18:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:18:02.173026 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221e6d2-3dd4-406e-a4c4-d52224ae27b6 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:02.287027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221e6d2-3dd4-406e-a4c4-d52224ae27b6 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:02.329062 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c53cd-5424-49d4-95b6-0e19c2d261c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:02.329790 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c53cd-5424-49d4-95b6-0e19c2d261c8 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:02.330110 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c53cd-5424-49d4-95b6-0e19c2d261c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:02.330410 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c53cd-5424-49d4-95b6-0e19c2d261c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:02.331165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9c53cd-5424-49d4-95b6-0e19c2d261c8 None admin] RBAC: Adding query filter params (role.id=bdf71ef6ab10430e8c356cd4d952ff5c, scope.project.id=9f4373dfb5f9425eb73669e24d18a08f, user.id=dd503fe3cf6d428e962c14ab864ee608) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:18:02.331470 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9c53cd-5424-49d4-95b6-0e19c2d261c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:02.333165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9c53cd-5424-49d4-95b6-0e19c2d261c8 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:02.340732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 623/1246] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:18:02 2022] GET /identity/v3/role_assignments?user.id=dd503fe3cf6d428e962c14ab864ee608&scope.project.id=9f4373dfb5f9425eb73669e24d18a08f&role.id=bdf71ef6ab10430e8c356cd4d952ff5c => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:18:04.444200 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f089826-9099-4493-aab4-ca180c911790 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:04.444878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f089826-9099-4493-aab4-ca180c911790 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:04.444878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f089826-9099-4493-aab4-ca180c911790 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:04.446193 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 624/1247] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:18:04.452257 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb725ef-37ef-4e84-80f2-76c047cea71e None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:04.452733 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb725ef-37ef-4e84-80f2-76c047cea71e None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:04.452818 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb725ef-37ef-4e84-80f2-76c047cea71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:04.459869 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:04.473041 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-cdb725ef-37ef-4e84-80f2-76c047cea71e None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:18:04.474339 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-cdb725ef-37ef-4e84-80f2-76c047cea71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:18:04.549478 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 624/1248] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:18:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:18:04.836594 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f089826-9099-4493-aab4-ca180c911790 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:04.856254 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:04.980932 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f089826-9099-4493-aab4-ca180c911790 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:05.018302 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d8ce3-a282-4d37-ab56-2b8ed296d647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:05.018937 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d8ce3-a282-4d37-ab56-2b8ed296d647 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:05.019211 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d8ce3-a282-4d37-ab56-2b8ed296d647 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:05.019540 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d8ce3-a282-4d37-ab56-2b8ed296d647 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:05.032165 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d8ce3-a282-4d37-ab56-2b8ed296d647 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:18:05.037027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-526d8ce3-a282-4d37-ab56-2b8ed296d647 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:18:05.037027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:18:05.037027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:18:05.037027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:18:05.037027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:18:05.037027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:18:05.037027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:18:05.037027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:18:05.037027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:18:05.037027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:18:05.037027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:18:05.037027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:18:05.037027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:18:05.037027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:18:05.037027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:18:05.037027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:18:05.038855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:18:05.038855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:18:05.038855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:18:05.038855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:18:05.038855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:18:05.038855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:18:05.038855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:18:05.038855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:18:05.038855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:18:05.038855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:18:05.038855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:18:05.038855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:18:05.038855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:18:05.038855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:18:05.038855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:18:05.038855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:18:05.040478 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:18:05.040478 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:18:05.040478 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:18:05.040478 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:18:05.040478 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:18:05.040478 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:18:05.040478 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:18:05.040478 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:18:05.040478 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:18:05.040478 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:18:05.040478 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:18:05.040478 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:18:05.043807 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 625/1249] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:18:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 04:18:05.044944 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb725ef-37ef-4e84-80f2-76c047cea71e None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:05.164207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb725ef-37ef-4e84-80f2-76c047cea71e None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:05.198128 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:05.203588 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41dfbc6-97ca-45e3-9dc4-4313239fecb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:05.204235 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41dfbc6-97ca-45e3-9dc4-4313239fecb8 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:05.204509 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41dfbc6-97ca-45e3-9dc4-4313239fecb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:05.204777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41dfbc6-97ca-45e3-9dc4-4313239fecb8 None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:05.205453 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41dfbc6-97ca-45e3-9dc4-4313239fecb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:18:05.207308 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41dfbc6-97ca-45e3-9dc4-4313239fecb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:05.207308 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41dfbc6-97ca-45e3-9dc4-4313239fecb8 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:05.217029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 625/1250] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:18:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:18:05.223255 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d8ce3-a282-4d37-ab56-2b8ed296d647 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:05.313756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:05.338698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d8ce3-a282-4d37-ab56-2b8ed296d647 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:05.377994 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4714dfbf-c7c3-4eef-8536-2493c9ea5b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:05.378640 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714dfbf-c7c3-4eef-8536-2493c9ea5b3f None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:05.378918 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714dfbf-c7c3-4eef-8536-2493c9ea5b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:05.379363 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714dfbf-c7c3-4eef-8536-2493c9ea5b3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:05.392886 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4714dfbf-c7c3-4eef-8536-2493c9ea5b3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:05.394708 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4714dfbf-c7c3-4eef-8536-2493c9ea5b3f None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:05.413225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-4714dfbf-c7c3-4eef-8536-2493c9ea5b3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:18:05.413225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:18:05.413225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:18:05.413225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:18:05.413225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:18:05.413225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:18:05.413225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:18:05.413225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:18:05.413225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:18:05.413225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:18:05.413225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:18:05.413225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:18:05.413225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:18:05.413225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:18:05.413225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:18:05.413225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:18:05.414318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:18:05.414318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:18:05.414318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:18:05.414318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:18:05.414318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:18:05.414318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:18:05.414318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:18:05.414318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:18:05.414318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:18:05.414318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:18:05.414318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:18:05.414318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:18:05.414318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:18:05.414318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:18:05.414318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:18:05.414318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:18:05.414318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:18:05.415202 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:18:05.415202 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:18:05.415202 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:18:05.415202 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 626/1251] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:18:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:18:05.421239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41dfbc6-97ca-45e3-9dc4-4313239fecb8 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:05.542240 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41dfbc6-97ca-45e3-9dc4-4313239fecb8 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:05.581782 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3dad6-518a-4eee-bbd0-39e274f2d86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:05.582399 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3dad6-518a-4eee-bbd0-39e274f2d86a None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:05.582677 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3dad6-518a-4eee-bbd0-39e274f2d86a None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:05.582949 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3dad6-518a-4eee-bbd0-39e274f2d86a None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:05.583598 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3dad6-518a-4eee-bbd0-39e274f2d86a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:18:05.584116 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3dad6-518a-4eee-bbd0-39e274f2d86a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:05.585553 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3dad6-518a-4eee-bbd0-39e274f2d86a None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:05.599389 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 626/1252] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:18:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:18:05.606626 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4714dfbf-c7c3-4eef-8536-2493c9ea5b3f None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:05.718405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4714dfbf-c7c3-4eef-8536-2493c9ea5b3f None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:05.746177 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:05.755201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0de49-9c26-412a-89e3-bac831633a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:05.755935 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0de49-9c26-412a-89e3-bac831633a4c None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:05.756234 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0de49-9c26-412a-89e3-bac831633a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:05.756507 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0de49-9c26-412a-89e3-bac831633a4c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:05.765030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e0de49-9c26-412a-89e3-bac831633a4c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:05.766615 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e0de49-9c26-412a-89e3-bac831633a4c None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:05.778321 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-84e0de49-9c26-412a-89e3-bac831633a4c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 04:18:05.778321 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:18:05.778321 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:18:05.778321 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:18:05.778321 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:18:05.778321 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:18:05.778321 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:18:05.778321 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:18:05.778321 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:18:05.778321 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:18:05.778321 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:18:05.778321 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:18:05.778321 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:18:05.778321 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:18:05.778321 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:18:05.778321 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:18:05.779554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:18:05.779554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:18:05.779554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:18:05.779554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:18:05.779554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:18:05.779554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:18:05.779554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:18:05.779554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:18:05.779554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:18:05.779554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:18:05.779554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:18:05.779554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:18:05.779554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:18:05.779554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:18:05.779554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:18:05.779554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:18:05.779554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:18:05.779554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:18:05.780872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:18:05.780872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:18:05.780872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:18:05.780872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:18:05.780872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:18:05.780872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:18:05.780872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:18:05.780872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:18:05.780872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 04:18:05.780872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:18:05.780872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 627/1253] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:18:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:18:05.785775 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3dad6-518a-4eee-bbd0-39e274f2d86a None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:05.846798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:05.904068 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3dad6-518a-4eee-bbd0-39e274f2d86a None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:05.949374 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a608e-f684-497b-b4ad-50a0a2670ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:05.950027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a608e-f684-497b-b4ad-50a0a2670ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:05.950306 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a608e-f684-497b-b4ad-50a0a2670ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:05.950576 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a608e-f684-497b-b4ad-50a0a2670ce9 None admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:05.951405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a608e-f684-497b-b4ad-50a0a2670ce9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:18:05.951750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a608e-f684-497b-b4ad-50a0a2670ce9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:05.953698 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a608e-f684-497b-b4ad-50a0a2670ce9 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:05.975672 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 627/1254] 10.222.0.18 () {56 vars in 1143 bytes} [Tue Aug 23 04:18:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:18:05.981912 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0de49-9c26-412a-89e3-bac831633a4c None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:06.102364 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0de49-9c26-412a-89e3-bac831633a4c None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:06.144697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa0889-20ed-42e1-9964-f9aa94fe4071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:06.145306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa0889-20ed-42e1-9964-f9aa94fe4071 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:06.145578 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa0889-20ed-42e1-9964-f9aa94fe4071 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:06.145843 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa0889-20ed-42e1-9964-f9aa94fe4071 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:06.210866 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:06.224829 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa0889-20ed-42e1-9964-f9aa94fe4071 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:06.226522 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa0889-20ed-42e1-9964-f9aa94fe4071 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:06.335627 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 628/1255] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:18:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 04:18:06.342587 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a608e-f684-497b-b4ad-50a0a2670ce9 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:06.454430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a608e-f684-497b-b4ad-50a0a2670ce9 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:06.469842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:06.494085 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959c4cf-c647-4715-9429-aad9917d474f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:06.494727 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959c4cf-c647-4715-9429-aad9917d474f None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:06.495106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959c4cf-c647-4715-9429-aad9917d474f None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:06.495408 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959c4cf-c647-4715-9429-aad9917d474f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:06.507693 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5959c4cf-c647-4715-9429-aad9917d474f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:06.509526 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5959c4cf-c647-4715-9429-aad9917d474f None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:06.525438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-5959c4cf-c647-4715-9429-aad9917d474f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:18:06.525438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:18:06.525438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:18:06.525438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:18:06.525438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:18:06.525438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:18:06.525438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:18:06.525438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:18:06.525438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:18:06.525438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:18:06.525438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:18:06.525438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:18:06.525438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:18:06.525438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:18:06.525438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:18:06.525438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:18:06.526464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:18:06.526464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:18:06.526464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:18:06.526464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:18:06.526464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:18:06.526464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:18:06.526464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:18:06.526464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:18:06.526464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:18:06.526464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:18:06.526464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:18:06.526464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:18:06.526464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:18:06.526464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:18:06.526464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:18:06.526464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:18:06.526464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:18:06.527543 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:18:06.527543 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:18:06.527543 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:18:06.527543 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 628/1256] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:18:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:18:06.533126 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa0889-20ed-42e1-9964-f9aa94fe4071 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:06.628381 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:06.643881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa0889-20ed-42e1-9964-f9aa94fe4071 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:06.680934 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c28de-d496-4487-9e40-fef5369338d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:06.681628 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c28de-d496-4487-9e40-fef5369338d6 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:06.681855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c28de-d496-4487-9e40-fef5369338d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:06.682179 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c28de-d496-4487-9e40-fef5369338d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:06.683027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c28de-d496-4487-9e40-fef5369338d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:18:06.683427 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c28de-d496-4487-9e40-fef5369338d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:06.684962 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c28de-d496-4487-9e40-fef5369338d6 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:06.697481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 629/1257] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:18:06.703235 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959c4cf-c647-4715-9429-aad9917d474f None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:06.822624 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959c4cf-c647-4715-9429-aad9917d474f None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:06.859538 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb0cee-1991-4f12-a784-b61b4671493f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:06.860203 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb0cee-1991-4f12-a784-b61b4671493f None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:06.860472 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb0cee-1991-4f12-a784-b61b4671493f None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:06.860744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb0cee-1991-4f12-a784-b61b4671493f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:06.868529 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb0cee-1991-4f12-a784-b61b4671493f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:06.870434 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb0cee-1991-4f12-a784-b61b4671493f None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:06.886927 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-3dfb0cee-1991-4f12-a784-b61b4671493f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:18:06.886927 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:18:06.886927 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:18:06.886927 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:18:06.886927 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:18:06.886927 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:18:06.886927 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:18:06.886927 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:18:06.886927 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:18:06.886927 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:18:06.886927 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:18:06.886927 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:18:06.886927 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:18:06.886927 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:18:06.886927 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:18:06.888179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:18:06.888179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:18:06.888179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:18:06.888179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:18:06.888179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:18:06.888179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:18:06.888179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:18:06.888179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:18:06.888179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:18:06.888179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:18:06.888179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:18:06.888179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:18:06.888179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:18:06.888179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:18:06.888179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:18:06.888179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:18:06.888179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:18:06.889422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:18:06.889422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:18:06.889422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:18:06.889422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:18:06.889422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:18:06.889422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:18:06.889422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:18:06.889422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:18:06.889422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 629/1258] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:18:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:18:06.895205 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c28de-d496-4487-9e40-fef5369338d6 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:07.023550 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c28de-d496-4487-9e40-fef5369338d6 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:07.062081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e449e-f08a-409a-bb95-0d4165501354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:07.062713 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e449e-f08a-409a-bb95-0d4165501354 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:07.062987 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e449e-f08a-409a-bb95-0d4165501354 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:07.063258 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e449e-f08a-409a-bb95-0d4165501354 None admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:07.063961 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e449e-f08a-409a-bb95-0d4165501354 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:18:07.064266 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e449e-f08a-409a-bb95-0d4165501354 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:07.065703 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e449e-f08a-409a-bb95-0d4165501354 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:07.079904 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 630/1259] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:18:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:18:07.086056 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb0cee-1991-4f12-a784-b61b4671493f None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:07.123538 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:07.207503 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb0cee-1991-4f12-a784-b61b4671493f None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:07.247201 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6bf721-1243-4603-b672-e898ea5719ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:07.247885 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6bf721-1243-4603-b672-e898ea5719ef None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:07.248192 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6bf721-1243-4603-b672-e898ea5719ef None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:07.248635 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6bf721-1243-4603-b672-e898ea5719ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:07.249436 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6bf721-1243-4603-b672-e898ea5719ef None admin] RBAC: Adding query filter params (role.id=52b31061b72c45ccb76dd26bf31f8d3f, scope.project.id=9f4373dfb5f9425eb73669e24d18a08f, user.id=dd503fe3cf6d428e962c14ab864ee608) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:18:07.249768 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6bf721-1243-4603-b672-e898ea5719ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:07.251542 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6bf721-1243-4603-b672-e898ea5719ef None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:07.258731 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 630/1260] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:18:07 2022] GET /identity/v3/role_assignments?user.id=dd503fe3cf6d428e962c14ab864ee608&scope.project.id=9f4373dfb5f9425eb73669e24d18a08f&role.id=52b31061b72c45ccb76dd26bf31f8d3f => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:18:09.188799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed34a5b-ed85-4ee3-8b90-9b228877cf13 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:09.189364 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed34a5b-ed85-4ee3-8b90-9b228877cf13 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:09.189364 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed34a5b-ed85-4ee3-8b90-9b228877cf13 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:09.190688 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 631/1261] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:18:09.197518 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea95249-efe7-4d4d-b075-82ac10ea907b None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:09.198309 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea95249-efe7-4d4d-b075-82ac10ea907b None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:09.198496 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea95249-efe7-4d4d-b075-82ac10ea907b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:09.227486 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-aea95249-efe7-4d4d-b075-82ac10ea907b None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:18:09.228706 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-aea95249-efe7-4d4d-b075-82ac10ea907b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:18:09.299296 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 631/1262] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:18:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:18:09.559858 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed34a5b-ed85-4ee3-8b90-9b228877cf13 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:09.572557 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:09.674665 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed34a5b-ed85-4ee3-8b90-9b228877cf13 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:09.711481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071c791-cff8-4d73-acfb-6154074a93ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:09.712058 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071c791-cff8-4d73-acfb-6154074a93ee None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:09.712336 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071c791-cff8-4d73-acfb-6154074a93ee None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:09.712607 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071c791-cff8-4d73-acfb-6154074a93ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:09.724267 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f071c791-cff8-4d73-acfb-6154074a93ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:18:09.728837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-f071c791-cff8-4d73-acfb-6154074a93ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:18:09.728837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:18:09.728837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:18:09.728837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:18:09.728837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:18:09.728837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:18:09.728837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:18:09.728837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:18:09.728837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:18:09.728837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:18:09.728837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:18:09.728837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:18:09.728837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:18:09.728837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:18:09.728837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:18:09.728837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:18:09.729927 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:18:09.729927 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:18:09.729927 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:18:09.729927 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:18:09.729927 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:18:09.729927 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:18:09.729927 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:18:09.729927 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:18:09.729927 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:18:09.729927 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:18:09.729927 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:18:09.729927 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:18:09.729927 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:18:09.729927 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:18:09.729927 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:18:09.729927 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:18:09.730745 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:18:09.730745 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:18:09.730745 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:18:09.730745 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:18:09.730745 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:18:09.730745 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:18:09.730745 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:18:09.730745 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:18:09.730745 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:18:09.730745 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:18:09.730745 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:18:09.730745 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:18:09.730745 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 632/1263] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:18:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:18:09.738596 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea95249-efe7-4d4d-b075-82ac10ea907b None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:09.759063 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:09.852557 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea95249-efe7-4d4d-b075-82ac10ea907b None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:09.891278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd2b81-420c-4818-85fc-fe5c270bb1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:09.892239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd2b81-420c-4818-85fc-fe5c270bb1e0 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:09.892799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd2b81-420c-4818-85fc-fe5c270bb1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:09.893200 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd2b81-420c-4818-85fc-fe5c270bb1e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:09.894333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd2b81-420c-4818-85fc-fe5c270bb1e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:18:09.894708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd2b81-420c-4818-85fc-fe5c270bb1e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:09.897684 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd2b81-420c-4818-85fc-fe5c270bb1e0 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:09.911266 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 632/1264] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:18:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:18:09.919369 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071c791-cff8-4d73-acfb-6154074a93ee None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:09.993314 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:10.032660 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071c791-cff8-4d73-acfb-6154074a93ee None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:10.077908 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f5deee-0ce9-41c6-be90-1a22dc5f0974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:10.078832 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5deee-0ce9-41c6-be90-1a22dc5f0974 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:10.079235 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5deee-0ce9-41c6-be90-1a22dc5f0974 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:10.079624 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5deee-0ce9-41c6-be90-1a22dc5f0974 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:10.097719 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f5deee-0ce9-41c6-be90-1a22dc5f0974 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:10.100177 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f5deee-0ce9-41c6-be90-1a22dc5f0974 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:10.116132 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-76f5deee-0ce9-41c6-be90-1a22dc5f0974 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:18:10.116132 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:18:10.116132 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:18:10.116132 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:18:10.116132 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:18:10.116132 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:18:10.116132 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:18:10.116132 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:18:10.116132 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:18:10.116132 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:18:10.116132 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:18:10.116132 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:18:10.116132 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:18:10.116132 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:18:10.116132 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:18:10.116132 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:18:10.117177 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:18:10.117177 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:18:10.117177 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:18:10.117177 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:18:10.117177 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:18:10.117177 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:18:10.117177 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:18:10.117177 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:18:10.117177 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:18:10.117177 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:18:10.117177 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:18:10.117177 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:18:10.117177 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:18:10.117177 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:18:10.117177 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:18:10.117177 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:18:10.117177 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:18:10.118397 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:18:10.118397 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:18:10.118397 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:18:10.118397 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 633/1265] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:18:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:18:10.125937 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd2b81-420c-4818-85fc-fe5c270bb1e0 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:10.213861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:10.234323 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd2b81-420c-4818-85fc-fe5c270bb1e0 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:10.270632 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62d736-099b-49bd-bc09-493e9d3a848f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:10.271158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62d736-099b-49bd-bc09-493e9d3a848f None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:10.271429 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62d736-099b-49bd-bc09-493e9d3a848f None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:10.271699 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62d736-099b-49bd-bc09-493e9d3a848f None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:10.272368 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac62d736-099b-49bd-bc09-493e9d3a848f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:18:10.272665 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac62d736-099b-49bd-bc09-493e9d3a848f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:10.274140 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac62d736-099b-49bd-bc09-493e9d3a848f None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:10.285851 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 633/1266] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:18:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:18:10.295002 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f5deee-0ce9-41c6-be90-1a22dc5f0974 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:10.398783 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f5deee-0ce9-41c6-be90-1a22dc5f0974 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:10.419699 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:10.434920 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ea155-0ba3-4b66-92e1-eeccd7512674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:10.435539 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ea155-0ba3-4b66-92e1-eeccd7512674 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:10.435811 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ea155-0ba3-4b66-92e1-eeccd7512674 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:10.436110 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ea155-0ba3-4b66-92e1-eeccd7512674 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:10.443374 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ea155-0ba3-4b66-92e1-eeccd7512674 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:10.444874 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ea155-0ba3-4b66-92e1-eeccd7512674 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:10.455712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-036ea155-0ba3-4b66-92e1-eeccd7512674 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 04:18:10.455712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:18:10.455712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:18:10.455712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:18:10.455712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:18:10.455712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:18:10.455712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:18:10.455712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:18:10.455712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:18:10.455712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:18:10.455712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:18:10.455712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:18:10.455712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:18:10.455712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:18:10.455712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:18:10.455712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:18:10.456697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:18:10.456697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:18:10.456697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:18:10.456697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:18:10.456697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:18:10.456697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:18:10.456697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:18:10.456697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:18:10.456697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:18:10.456697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:18:10.456697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:18:10.456697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:18:10.456697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:18:10.456697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:18:10.456697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:18:10.456697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:18:10.456697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:18:10.456697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:18:10.458111 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:18:10.458111 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:18:10.458111 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:18:10.458111 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:18:10.458111 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:18:10.458111 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:18:10.458111 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:18:10.458111 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:18:10.458111 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 04:18:10.458111 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:18:10.458111 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 634/1267] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:18:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:18:10.464081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62d736-099b-49bd-bc09-493e9d3a848f None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:10.570805 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62d736-099b-49bd-bc09-493e9d3a848f None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:10.607603 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a002d3f-9dac-4c67-944a-32350be95df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:10.608232 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a002d3f-9dac-4c67-944a-32350be95df0 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:10.608506 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a002d3f-9dac-4c67-944a-32350be95df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:10.608771 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a002d3f-9dac-4c67-944a-32350be95df0 None admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:10.609477 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a002d3f-9dac-4c67-944a-32350be95df0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:18:10.609778 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a002d3f-9dac-4c67-944a-32350be95df0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:10.611213 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a002d3f-9dac-4c67-944a-32350be95df0 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:10.621380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:10.630997 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 634/1268] 10.222.0.18 () {56 vars in 1143 bytes} [Tue Aug 23 04:18:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:18:10.637807 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ea155-0ba3-4b66-92e1-eeccd7512674 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:10.753309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ea155-0ba3-4b66-92e1-eeccd7512674 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:10.788562 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1c091-ee57-410e-8c9a-d75ed6932614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:10.789177 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1c091-ee57-410e-8c9a-d75ed6932614 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:10.789451 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1c091-ee57-410e-8c9a-d75ed6932614 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:10.789744 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1c091-ee57-410e-8c9a-d75ed6932614 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:10.800196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f1c091-ee57-410e-8c9a-d75ed6932614 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:10.801687 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f1c091-ee57-410e-8c9a-d75ed6932614 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:10.825350 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-21f1c091-ee57-410e-8c9a-d75ed6932614 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:18:10.825350 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:18:10.825350 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:18:10.825350 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:18:10.825350 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:18:10.825350 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:18:10.825350 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:18:10.825350 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:18:10.825350 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:18:10.825350 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:18:10.825350 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:18:10.825350 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:18:10.825350 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:18:10.825350 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:18:10.825350 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:18:10.825350 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:18:10.826429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:18:10.826429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:18:10.826429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:18:10.826429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:18:10.826429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:18:10.826429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:18:10.826429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:18:10.826429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:18:10.826429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:18:10.826429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:18:10.826429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:18:10.826429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:18:10.826429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:18:10.826429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:18:10.826429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:18:10.826429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:18:10.826429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:18:10.827408 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:18:10.827408 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:18:10.827408 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:18:10.827408 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 635/1269] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:18:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:18:10.834717 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a002d3f-9dac-4c67-944a-32350be95df0 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:10.969898 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a002d3f-9dac-4c67-944a-32350be95df0 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:11.005201 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b03dd-f81e-4bb1-8632-9eb5a9c94cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:11.005802 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b03dd-f81e-4bb1-8632-9eb5a9c94cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:11.006081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b03dd-f81e-4bb1-8632-9eb5a9c94cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:11.006348 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b03dd-f81e-4bb1-8632-9eb5a9c94cc8 None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:11.007193 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b03dd-f81e-4bb1-8632-9eb5a9c94cc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:18:11.007398 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b03dd-f81e-4bb1-8632-9eb5a9c94cc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:11.008808 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b03dd-f81e-4bb1-8632-9eb5a9c94cc8 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:11.020088 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 635/1270] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:18:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:18:11.026902 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1c091-ee57-410e-8c9a-d75ed6932614 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:11.065427 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:11.144431 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1c091-ee57-410e-8c9a-d75ed6932614 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:11.191550 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de780d37-1042-412a-a30a-aa84fe6fec3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:11.192527 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de780d37-1042-412a-a30a-aa84fe6fec3a None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:11.192915 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de780d37-1042-412a-a30a-aa84fe6fec3a None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:11.193290 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de780d37-1042-412a-a30a-aa84fe6fec3a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:11.200830 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de780d37-1042-412a-a30a-aa84fe6fec3a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:11.202284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de780d37-1042-412a-a30a-aa84fe6fec3a None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:11.221891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-de780d37-1042-412a-a30a-aa84fe6fec3a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:18:11.221891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:18:11.221891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:18:11.221891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:18:11.221891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:18:11.221891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:18:11.221891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:18:11.221891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:18:11.221891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:18:11.221891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:18:11.221891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:18:11.221891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:18:11.221891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:18:11.221891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:18:11.221891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:18:11.223049 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:18:11.223049 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:18:11.223049 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:18:11.223049 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:18:11.223049 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:18:11.223049 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:18:11.223049 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:18:11.223049 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:18:11.223049 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:18:11.223049 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:18:11.223049 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:18:11.223049 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:18:11.223049 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:18:11.223049 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:18:11.223049 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:18:11.223049 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:18:11.223049 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:18:11.224072 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:18:11.224072 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:18:11.224072 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:18:11.224072 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:18:11.224072 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:18:11.224072 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:18:11.224072 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:18:11.224072 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:18:11.224072 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 636/1271] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:18:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:18:11.230605 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b03dd-f81e-4bb1-8632-9eb5a9c94cc8 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:11.305398 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:11.350203 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b03dd-f81e-4bb1-8632-9eb5a9c94cc8 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:11.386364 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3199163f-5cf2-49ff-b054-87d84d76ba23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:11.386964 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199163f-5cf2-49ff-b054-87d84d76ba23 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:11.387241 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199163f-5cf2-49ff-b054-87d84d76ba23 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:11.387513 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199163f-5cf2-49ff-b054-87d84d76ba23 None admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:11.388237 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3199163f-5cf2-49ff-b054-87d84d76ba23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:18:11.388538 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3199163f-5cf2-49ff-b054-87d84d76ba23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:11.389978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3199163f-5cf2-49ff-b054-87d84d76ba23 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:11.402428 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 636/1272] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:18:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:18:11.408793 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de780d37-1042-412a-a30a-aa84fe6fec3a None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:11.509248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:11.524947 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de780d37-1042-412a-a30a-aa84fe6fec3a None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:11.560504 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2732e6-677b-400d-bd5e-2b0a55164bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:11.561153 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2732e6-677b-400d-bd5e-2b0a55164bbf None admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:11.561430 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2732e6-677b-400d-bd5e-2b0a55164bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:11.561714 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2732e6-677b-400d-bd5e-2b0a55164bbf None admin] PATH_INFO: `/v3/projects/9f4373dfb5f9425eb73669e24d18a08f/users/dd503fe3cf6d428e962c14ab864ee608/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:11.563706 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2732e6-677b-400d-bd5e-2b0a55164bbf None admin] RBAC: Authorizing `identity:create_grant(project_id=9f4373dfb5f9425eb73669e24d18a08f, user_id=dd503fe3cf6d428e962c14ab864ee608, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:11.565182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2732e6-677b-400d-bd5e-2b0a55164bbf None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:11.582584 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 637/1273] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Aug 23 04:18:11 2022] PUT /identity/v3/projects/9f4373dfb5f9425eb73669e24d18a08f/users/dd503fe3cf6d428e962c14ab864ee608/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:18:13.433078 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e8d62-6dff-452b-8437-cbc2f6dfb8db None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:13.434016 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e8d62-6dff-452b-8437-cbc2f6dfb8db None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:13.434016 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e8d62-6dff-452b-8437-cbc2f6dfb8db None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:13.436178 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 637/1274] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:18:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:18:13.442588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fef43b-396d-45da-b218-bc58632cb171 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:13.443326 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fef43b-396d-45da-b218-bc58632cb171 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:13.443480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fef43b-396d-45da-b218-bc58632cb171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:13.463649 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-92fef43b-396d-45da-b218-bc58632cb171 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:18:13.464891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-92fef43b-396d-45da-b218-bc58632cb171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:18:13.534984 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 638/1275] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:18:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:18:13.805227 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e8d62-6dff-452b-8437-cbc2f6dfb8db None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:13.930561 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e8d62-6dff-452b-8437-cbc2f6dfb8db None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:13.948386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:13.977570 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8957a2-4855-471f-9af8-b6d2d40e9ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:13.978199 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8957a2-4855-471f-9af8-b6d2d40e9ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:13.978475 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8957a2-4855-471f-9af8-b6d2d40e9ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:13.978743 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8957a2-4855-471f-9af8-b6d2d40e9ef5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:13.989189 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8957a2-4855-471f-9af8-b6d2d40e9ef5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 04:18:13.993952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-2f8957a2-4855-471f-9af8-b6d2d40e9ef5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:18:13.993952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:18:13.993952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:18:13.993952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:18:13.993952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:18:13.993952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:18:13.993952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:18:13.993952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:18:13.993952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:18:13.993952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:18:13.993952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:18:13.993952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:18:13.993952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 04:18:13.993952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 04:18:13.993952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 04:18:13.993952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 04:18:13.994988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 04:18:13.994988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 04:18:13.994988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 04:18:13.994988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 04:18:13.994988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:18:13.994988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:18:13.994988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:18:13.994988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:18:13.994988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:18:13.994988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:18:13.994988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:18:13.994988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:18:13.994988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:18:13.994988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:18:13.994988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:18:13.994988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:18:13.995880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:18:13.995880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:18:13.995880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:18:13.995880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:18:13.995880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 04:18:13.995880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 04:18:13.995880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 04:18:13.995880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 04:18:13.995880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 04:18:13.995880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 04:18:13.995880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 04:18:13.995880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:18:13.995880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 638/1276] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Aug 23 04:18:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:18:14.001666 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fef43b-396d-45da-b218-bc58632cb171 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:14.088838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:14.110682 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fef43b-396d-45da-b218-bc58632cb171 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:14.145149 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38df15-fb0f-446c-8f9c-b4650d96c513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:14.145754 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38df15-fb0f-446c-8f9c-b4650d96c513 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:14.146032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38df15-fb0f-446c-8f9c-b4650d96c513 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:14.146295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38df15-fb0f-446c-8f9c-b4650d96c513 None admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:14.146953 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38df15-fb0f-446c-8f9c-b4650d96c513 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:18:14.147249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38df15-fb0f-446c-8f9c-b4650d96c513 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:14.148671 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38df15-fb0f-446c-8f9c-b4650d96c513 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:14.156868 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 639/1277] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Aug 23 04:18:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:18:14.162783 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8957a2-4855-471f-9af8-b6d2d40e9ef5 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:14.278482 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8957a2-4855-471f-9af8-b6d2d40e9ef5 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:14.313602 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dbc8d8-8a97-4715-bf54-9d6334e16e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:14.314209 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbc8d8-8a97-4715-bf54-9d6334e16e87 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:14.314481 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbc8d8-8a97-4715-bf54-9d6334e16e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:14.314752 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbc8d8-8a97-4715-bf54-9d6334e16e87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:14.327428 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dbc8d8-8a97-4715-bf54-9d6334e16e87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:14.329633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dbc8d8-8a97-4715-bf54-9d6334e16e87 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:14.352153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-44dbc8d8-8a97-4715-bf54-9d6334e16e87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:18:14.352153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:18:14.352153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:18:14.352153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:18:14.352153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:18:14.352153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:18:14.352153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:18:14.352153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:18:14.352153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:18:14.352153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:18:14.352153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:18:14.352153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:18:14.352153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:18:14.352153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:18:14.352153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:18:14.352153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:18:14.353287 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:18:14.353287 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:18:14.353287 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:18:14.353287 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:18:14.353287 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:18:14.353287 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:18:14.353287 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:18:14.353287 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:18:14.353287 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:18:14.353287 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:18:14.353287 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:18:14.353287 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:18:14.353287 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:18:14.353287 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:18:14.353287 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:18:14.353287 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:18:14.353287 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:18:14.355057 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:18:14.355057 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:18:14.355057 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:18:14.355057 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 639/1278] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:18:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:18:14.362238 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38df15-fb0f-446c-8f9c-b4650d96c513 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:14.468222 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38df15-fb0f-446c-8f9c-b4650d96c513 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:14.504648 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba35ff4-eb73-41de-b9c5-e259db632316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:14.505249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba35ff4-eb73-41de-b9c5-e259db632316 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:14.505495 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba35ff4-eb73-41de-b9c5-e259db632316 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:14.505786 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba35ff4-eb73-41de-b9c5-e259db632316 None admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:14.506418 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba35ff4-eb73-41de-b9c5-e259db632316 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:18:14.506714 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba35ff4-eb73-41de-b9c5-e259db632316 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:14.508251 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba35ff4-eb73-41de-b9c5-e259db632316 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:14.519981 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 640/1279] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:18:14.526189 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dbc8d8-8a97-4715-bf54-9d6334e16e87 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:14.561208 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:14.634894 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dbc8d8-8a97-4715-bf54-9d6334e16e87 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:14.670912 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f562bd2-bbce-46e9-a1d5-bd172492380f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:14.671525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f562bd2-bbce-46e9-a1d5-bd172492380f None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:14.672129 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f562bd2-bbce-46e9-a1d5-bd172492380f None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:14.672369 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f562bd2-bbce-46e9-a1d5-bd172492380f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:14.680522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f562bd2-bbce-46e9-a1d5-bd172492380f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:14.682746 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f562bd2-bbce-46e9-a1d5-bd172492380f None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:14.699881 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-1f562bd2-bbce-46e9-a1d5-bd172492380f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 04:18:14.699881 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:18:14.699881 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:18:14.699881 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:18:14.699881 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:18:14.699881 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:18:14.699881 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:18:14.699881 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:18:14.699881 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:18:14.699881 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:18:14.699881 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:18:14.699881 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:18:14.699881 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 04:18:14.699881 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 04:18:14.699881 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 04:18:14.699881 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 04:18:14.701022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:18:14.701022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:18:14.701022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 04:18:14.701022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:18:14.701022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 04:18:14.701022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 04:18:14.701022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:18:14.701022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:18:14.701022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:18:14.701022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:18:14.701022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:18:14.701022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:18:14.701022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:18:14.701022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:18:14.701022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:18:14.701022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:18:14.701022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:18:14.701022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:18:14.702392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:18:14.702392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:18:14.702392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 04:18:14.702392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 04:18:14.702392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 04:18:14.702392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 04:18:14.702392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 04:18:14.702392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 04:18:14.702392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 04:18:14.702392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:18:14.702392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 640/1280] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:18:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:18:14.707895 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba35ff4-eb73-41de-b9c5-e259db632316 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:14.715362 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:14.821369 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba35ff4-eb73-41de-b9c5-e259db632316 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:14.869782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24e25fd-c33e-4c82-984b-6a571546a146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:14.870529 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e25fd-c33e-4c82-984b-6a571546a146 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:14.870860 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e25fd-c33e-4c82-984b-6a571546a146 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:14.871151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e25fd-c33e-4c82-984b-6a571546a146 None admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:14.872094 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24e25fd-c33e-4c82-984b-6a571546a146 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:18:14.872410 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24e25fd-c33e-4c82-984b-6a571546a146 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:14.874052 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24e25fd-c33e-4c82-984b-6a571546a146 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:14.896726 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 641/1281] 10.222.0.18 () {56 vars in 1143 bytes} [Tue Aug 23 04:18:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:18:14.903281 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f562bd2-bbce-46e9-a1d5-bd172492380f None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:15.001177 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:15.011776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f562bd2-bbce-46e9-a1d5-bd172492380f None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:15.048187 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb9a0a1-b849-4552-a00d-ae43fe13d86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:15.048782 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9a0a1-b849-4552-a00d-ae43fe13d86b None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:15.049060 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9a0a1-b849-4552-a00d-ae43fe13d86b None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:15.049330 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9a0a1-b849-4552-a00d-ae43fe13d86b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:15.120584 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb9a0a1-b849-4552-a00d-ae43fe13d86b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:15.122032 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb9a0a1-b849-4552-a00d-ae43fe13d86b None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:15.218823 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:15.235468 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 641/1282] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Aug 23 04:18:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 04:18:15.243947 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24e25fd-c33e-4c82-984b-6a571546a146 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:15.323887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:15.364524 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24e25fd-c33e-4c82-984b-6a571546a146 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:15.399571 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9ff55-7e35-47c5-96d9-485c2591ca1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:15.400227 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9ff55-7e35-47c5-96d9-485c2591ca1e None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:15.400501 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9ff55-7e35-47c5-96d9-485c2591ca1e None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:15.400759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9ff55-7e35-47c5-96d9-485c2591ca1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:15.411208 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd9ff55-7e35-47c5-96d9-485c2591ca1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:15.412684 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd9ff55-7e35-47c5-96d9-485c2591ca1e None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:15.427027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-3bd9ff55-7e35-47c5-96d9-485c2591ca1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:18:15.427027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:18:15.427027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:18:15.427027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:18:15.427027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:18:15.427027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:18:15.427027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:18:15.427027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:18:15.427027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:18:15.427027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:18:15.427027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:18:15.427027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:18:15.427027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 04:18:15.427027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 04:18:15.427027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 04:18:15.427027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 04:18:15.430881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:18:15.430881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:18:15.430881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:18:15.430881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:18:15.430881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:18:15.430881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:18:15.430881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:18:15.430881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:18:15.430881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:18:15.430881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:18:15.430881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:18:15.430881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:18:15.430881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:18:15.430881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:18:15.430881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:18:15.430881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:18:15.430881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 04:18:15.432039 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 04:18:15.432039 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 04:18:15.432039 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:18:15.432039 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 642/1283] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:18:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:18:15.434637 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb9a0a1-b849-4552-a00d-ae43fe13d86b None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:15.541953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb9a0a1-b849-4552-a00d-ae43fe13d86b None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:15.577359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf22e75-28cd-4523-9d9f-d3fa6cc3837a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:15.577950 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf22e75-28cd-4523-9d9f-d3fa6cc3837a None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:15.578221 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf22e75-28cd-4523-9d9f-d3fa6cc3837a None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:15.578489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf22e75-28cd-4523-9d9f-d3fa6cc3837a None admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:15.579117 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf22e75-28cd-4523-9d9f-d3fa6cc3837a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:18:15.579418 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf22e75-28cd-4523-9d9f-d3fa6cc3837a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:15.580804 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf22e75-28cd-4523-9d9f-d3fa6cc3837a None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:15.592517 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 642/1284] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Aug 23 04:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:18:15.599117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9ff55-7e35-47c5-96d9-485c2591ca1e None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:15.708508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9ff55-7e35-47c5-96d9-485c2591ca1e None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:15.722297 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:15.742750 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc3dc3-dbd9-4cdb-acfe-4f7057e25ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:15.743372 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc3dc3-dbd9-4cdb-acfe-4f7057e25ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:15.743649 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc3dc3-dbd9-4cdb-acfe-4f7057e25ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:15.743941 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc3dc3-dbd9-4cdb-acfe-4f7057e25ff6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:15.750351 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acc3dc3-dbd9-4cdb-acfe-4f7057e25ff6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:15.751801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acc3dc3-dbd9-4cdb-acfe-4f7057e25ff6 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:15.762930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application [None req-0acc3dc3-dbd9-4cdb-acfe-4f7057e25ff6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:18:15.762930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:18:15.762930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:18:15.762930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:18:15.762930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:18:15.762930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:18:15.762930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:18:15.762930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:18:15.762930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:18:15.762930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:18:15.762930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:18:15.762930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:18:15.762930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 04:18:15.762930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 04:18:15.762930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 04:18:15.764073 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 04:18:15.764073 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:18:15.764073 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:18:15.764073 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:18:15.764073 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:18:15.764073 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:18:15.764073 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:18:15.764073 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:18:15.764073 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application with Lock( Aug 23 04:18:15.764073 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:18:15.764073 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:18:15.764073 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:18:15.764073 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:18:15.764073 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:18:15.764073 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:18:15.764073 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:18:15.764073 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:18:15.765573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 04:18:15.765573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 04:18:15.765573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 04:18:15.765573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 04:18:15.765573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 04:18:15.765573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 04:18:15.765573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 04:18:15.765573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: ERROR keystone.server.flask.application Aug 23 04:18:15.765573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 643/1285] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Aug 23 04:18:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:18:15.773988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf22e75-28cd-4523-9d9f-d3fa6cc3837a None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:15.831879 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:15.889491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf22e75-28cd-4523-9d9f-d3fa6cc3837a None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:15.929373 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f59e01-e8a6-4d27-bd9c-0d6d40e814ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:15.929994 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f59e01-e8a6-4d27-bd9c-0d6d40e814ee None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:15.930275 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f59e01-e8a6-4d27-bd9c-0d6d40e814ee None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:15.930551 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f59e01-e8a6-4d27-bd9c-0d6d40e814ee None admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:15.931245 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f59e01-e8a6-4d27-bd9c-0d6d40e814ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:18:15.931550 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f59e01-e8a6-4d27-bd9c-0d6d40e814ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:15.933027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f59e01-e8a6-4d27-bd9c-0d6d40e814ee None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:15.945678 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 643/1286] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Aug 23 04:18:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:18:15.951743 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc3dc3-dbd9-4cdb-acfe-4f7057e25ff6 None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:16.059157 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc3dc3-dbd9-4cdb-acfe-4f7057e25ff6 None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:16.093582 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c5586f-917f-47f7-ab8d-472c03296113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:16.094112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5586f-917f-47f7-ab8d-472c03296113 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:16.094390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5586f-917f-47f7-ab8d-472c03296113 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:16.094659 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5586f-917f-47f7-ab8d-472c03296113 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:16.095383 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c5586f-917f-47f7-ab8d-472c03296113 None admin] RBAC: Adding query filter params (role.id=52b31061b72c45ccb76dd26bf31f8d3f, scope.project.id=9f4373dfb5f9425eb73669e24d18a08f, user.id=dd503fe3cf6d428e962c14ab864ee608) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:18:16.095687 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c5586f-917f-47f7-ab8d-472c03296113 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:16.097177 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c5586f-917f-47f7-ab8d-472c03296113 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:16.103576 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 644/1287] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Aug 23 04:18:15 2022] GET /identity/v3/role_assignments?user.id=dd503fe3cf6d428e962c14ab864ee608&scope.project.id=9f4373dfb5f9425eb73669e24d18a08f&role.id=52b31061b72c45ccb76dd26bf31f8d3f => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:18:18.007900 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d171b7-7d85-4164-abc1-57aebefe3a8f None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:18.013014 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d171b7-7d85-4164-abc1-57aebefe3a8f None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:18.013014 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d171b7-7d85-4164-abc1-57aebefe3a8f None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:18.013014 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 644/1288] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:18:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:18:18.016499 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4366d7-1c8b-46d7-8195-4183303f4e56 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:18.016939 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4366d7-1c8b-46d7-8195-4183303f4e56 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:18.017037 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4366d7-1c8b-46d7-8195-4183303f4e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:18.037083 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-7e4366d7-1c8b-46d7-8195-4183303f4e56 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:18:18.038338 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-7e4366d7-1c8b-46d7-8195-4183303f4e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:18:18.082058 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:18.133158 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 645/1289] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:18:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:18:18.461588 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d171b7-7d85-4164-abc1-57aebefe3a8f None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:18.591457 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d171b7-7d85-4164-abc1-57aebefe3a8f None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:18.602867 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:18.631064 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a65b9-fa6d-4bc2-bc89-91c86d6509e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:18.631698 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a65b9-fa6d-4bc2-bc89-91c86d6509e0 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:18.632126 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a65b9-fa6d-4bc2-bc89-91c86d6509e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:18.632404 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a65b9-fa6d-4bc2-bc89-91c86d6509e0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:18.633002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04a65b9-fa6d-4bc2-bc89-91c86d6509e0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:18.634405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04a65b9-fa6d-4bc2-bc89-91c86d6509e0 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:18.644081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-b04a65b9-fa6d-4bc2-bc89-91c86d6509e0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 04:18:18.644081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:18:18.644081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 04:18:18.644081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 04:18:18.644081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 04:18:18.644081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 04:18:18.644081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 04:18:18.644081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:18:18.644081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 04:18:18.644081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:18:18.644081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:18:18.644081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:18:18.644081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:18:18.644081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:18:18.644081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:18:18.644081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:18:18.644081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:18:18.645263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:18:18.645263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:18:18.645263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:18:18.645263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:18:18.645263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:18:18.645263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:18:18.645263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 04:18:18.645263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 04:18:18.645263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 04:18:18.645263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 04:18:18.645263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:18:18.645263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:18:18.645263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:18:18.645263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:18:18.645263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:18:18.645263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:18:18.646346 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:18:18.646346 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:18:18.646346 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:18:18.646346 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:18:18.646346 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:18:18.646346 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:18:18.646346 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:18:18.646346 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:18:18.646346 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:18:18.646346 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:18:18.646346 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 04:18:18.646346 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:18:18.646346 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 04:18:18.646346 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:18:18.646346 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 645/1290] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Aug 23 04:18:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:18:18.651425 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4366d7-1c8b-46d7-8195-4183303f4e56 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:18.768830 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4366d7-1c8b-46d7-8195-4183303f4e56 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:18.804818 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cde7e1-86b2-4585-8a46-2f105f7e3540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:18.805815 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cde7e1-86b2-4585-8a46-2f105f7e3540 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:18.806097 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cde7e1-86b2-4585-8a46-2f105f7e3540 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:18.806371 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cde7e1-86b2-4585-8a46-2f105f7e3540 None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:18.807011 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cde7e1-86b2-4585-8a46-2f105f7e3540 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:18:18.807313 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cde7e1-86b2-4585-8a46-2f105f7e3540 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:18.808747 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cde7e1-86b2-4585-8a46-2f105f7e3540 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:18.813771 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 646/1291] 10.222.0.18 () {56 vars in 1116 bytes} [Tue Aug 23 04:18:18 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:18:18.819953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a65b9-fa6d-4bc2-bc89-91c86d6509e0 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:18.936954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a65b9-fa6d-4bc2-bc89-91c86d6509e0 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:18.988856 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a703c2-5327-499a-9a10-b2303e87a0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:18.989957 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a703c2-5327-499a-9a10-b2303e87a0e0 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:18.990673 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a703c2-5327-499a-9a10-b2303e87a0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:18.992569 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a703c2-5327-499a-9a10-b2303e87a0e0 None admin] PATH_INFO: `/v3/services` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:18.993316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a703c2-5327-499a-9a10-b2303e87a0e0 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:18:18.993657 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a703c2-5327-499a-9a10-b2303e87a0e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:18.996086 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a703c2-5327-499a-9a10-b2303e87a0e0 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:19.003538 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 646/1292] 10.222.0.18 () {56 vars in 1116 bytes} [Tue Aug 23 04:18:18 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:18:20.897936 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97409b-fe2a-46fd-932d-5bd86651d045 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:20.898844 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97409b-fe2a-46fd-932d-5bd86651d045 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:20.899260 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97409b-fe2a-46fd-932d-5bd86651d045 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:20.901481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 647/1293] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:18:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:18:20.910080 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d44e4-b136-45dd-a157-bb9a065b5f4c None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:20.910496 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d44e4-b136-45dd-a157-bb9a065b5f4c None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:20.910658 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d44e4-b136-45dd-a157-bb9a065b5f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:20.930618 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-e01d44e4-b136-45dd-a157-bb9a065b5f4c None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:18:20.931799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-e01d44e4-b136-45dd-a157-bb9a065b5f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:18:20.962552 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:21.002822 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 647/1294] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:18:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:18:21.275133 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97409b-fe2a-46fd-932d-5bd86651d045 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:21.320798 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:21.396935 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97409b-fe2a-46fd-932d-5bd86651d045 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:21.432569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042b544-748a-4e6b-8635-710c7a0f1823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:21.433181 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042b544-748a-4e6b-8635-710c7a0f1823 None admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:21.433457 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042b544-748a-4e6b-8635-710c7a0f1823 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:21.433726 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042b544-748a-4e6b-8635-710c7a0f1823 None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:21.434364 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2042b544-748a-4e6b-8635-710c7a0f1823 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:21.435767 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2042b544-748a-4e6b-8635-710c7a0f1823 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:21.453912 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 648/1295] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Aug 23 04:18:21 2022] POST /identity/v3/services => generated 246 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:18:23.339949 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9fdcc-2733-4364-a98c-fff32d5227d8 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:23.340547 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9fdcc-2733-4364-a98c-fff32d5227d8 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:23.340547 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9fdcc-2733-4364-a98c-fff32d5227d8 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:23.341852 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 648/1296] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:18:23.348399 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00076566-5ce9-4416-adeb-426566d44ac7 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:23.349078 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00076566-5ce9-4416-adeb-426566d44ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:23.349297 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00076566-5ce9-4416-adeb-426566d44ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:23.374647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-00076566-5ce9-4416-adeb-426566d44ac7 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:18:23.375856 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-00076566-5ce9-4416-adeb-426566d44ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:18:23.466298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 649/1297] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:18:23 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:18:23.749441 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9fdcc-2733-4364-a98c-fff32d5227d8 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:23.885842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9fdcc-2733-4364-a98c-fff32d5227d8 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:23.924121 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2cc63e-9450-48b9-82d8-3c995d0b569b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:23.924740 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2cc63e-9450-48b9-82d8-3c995d0b569b None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:23.925008 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2cc63e-9450-48b9-82d8-3c995d0b569b None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:23.925276 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2cc63e-9450-48b9-82d8-3c995d0b569b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:23.925853 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2cc63e-9450-48b9-82d8-3c995d0b569b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:23.927210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2cc63e-9450-48b9-82d8-3c995d0b569b None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:23.936383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-af2cc63e-9450-48b9-82d8-3c995d0b569b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 04:18:23.936383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:18:23.936383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 04:18:23.936383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 04:18:23.936383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 04:18:23.936383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 04:18:23.936383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 04:18:23.936383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:18:23.936383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 04:18:23.936383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:18:23.936383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:18:23.936383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:18:23.936383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:18:23.936383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:18:23.936383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:18:23.936383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:18:23.936383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:18:23.940096 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:18:23.940096 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:18:23.940096 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:18:23.940096 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:18:23.940096 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:18:23.940096 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:18:23.940096 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 04:18:23.940096 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 04:18:23.940096 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 04:18:23.940096 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 04:18:23.940096 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:18:23.940096 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:18:23.940096 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:18:23.940096 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:18:23.940096 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:18:23.940096 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:18:23.941129 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:18:23.941129 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:18:23.941129 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:18:23.941129 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:18:23.941129 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:18:23.941129 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:18:23.941129 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:18:23.941129 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:18:23.941129 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:18:23.941129 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:18:23.941129 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 04:18:23.941129 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:18:23.941129 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 04:18:23.941129 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:18:23.941129 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 649/1298] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Aug 23 04:18:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:18:23.945081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00076566-5ce9-4416-adeb-426566d44ac7 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:23.965325 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:24.046650 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00076566-5ce9-4416-adeb-426566d44ac7 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:24.082090 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4f9d8-91f3-40c3-84f8-5d5c757bc5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:24.082685 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4f9d8-91f3-40c3-84f8-5d5c757bc5aa None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:24.082959 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4f9d8-91f3-40c3-84f8-5d5c757bc5aa None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:24.083226 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4f9d8-91f3-40c3-84f8-5d5c757bc5aa None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:24.083849 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4f9d8-91f3-40c3-84f8-5d5c757bc5aa None admin] RBAC: Adding query filter params (name=placement) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:18:24.084182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4f9d8-91f3-40c3-84f8-5d5c757bc5aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:24.085618 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4f9d8-91f3-40c3-84f8-5d5c757bc5aa None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:24.091155 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 650/1299] 10.222.0.18 () {56 vars in 1116 bytes} [Tue Aug 23 04:18:23 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:18:24.101126 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2cc63e-9450-48b9-82d8-3c995d0b569b None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:24.140371 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:24.226041 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2cc63e-9450-48b9-82d8-3c995d0b569b None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:24.266363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf173e-0030-4c0f-a11e-561dca82b99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:24.266969 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf173e-0030-4c0f-a11e-561dca82b99c None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:24.267247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf173e-0030-4c0f-a11e-561dca82b99c None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:24.267516 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf173e-0030-4c0f-a11e-561dca82b99c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:24.268256 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf173e-0030-4c0f-a11e-561dca82b99c None admin] RBAC: Adding query filter params (interface=public, service_id=48ac8b79a94c435683cb1e741e41f41e, region_id=RegionOne) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:18:24.268564 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf173e-0030-4c0f-a11e-561dca82b99c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:24.269943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf173e-0030-4c0f-a11e-561dca82b99c None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:24.275721 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 650/1300] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Aug 23 04:18:24 2022] GET /identity/v3/endpoints?service_id=48ac8b79a94c435683cb1e741e41f41e&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:18:24.281606 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4f9d8-91f3-40c3-84f8-5d5c757bc5aa None admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:24.367809 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:24.390971 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4f9d8-91f3-40c3-84f8-5d5c757bc5aa None admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:24.427290 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7507193-8e3e-48d4-ba9b-da720015d728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:24.427886 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7507193-8e3e-48d4-ba9b-da720015d728 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:24.428199 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7507193-8e3e-48d4-ba9b-da720015d728 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:24.428466 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7507193-8e3e-48d4-ba9b-da720015d728 None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:24.429099 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7507193-8e3e-48d4-ba9b-da720015d728 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:24.430463 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7507193-8e3e-48d4-ba9b-da720015d728 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:24.437221 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 651/1301] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Aug 23 04:18:24 2022] GET /identity/v3/services => generated 2191 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:18:26.349041 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f56716-4399-4631-b61a-15691f251707 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:26.349896 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f56716-4399-4631-b61a-15691f251707 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:26.349896 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f56716-4399-4631-b61a-15691f251707 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:26.352089 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 651/1302] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:18:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:18:26.357450 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a496888-94d9-4245-9bf6-8b829ddb1dcf None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:26.357866 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a496888-94d9-4245-9bf6-8b829ddb1dcf None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:26.357953 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a496888-94d9-4245-9bf6-8b829ddb1dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:26.384533 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-7a496888-94d9-4245-9bf6-8b829ddb1dcf None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:18:26.386343 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-7a496888-94d9-4245-9bf6-8b829ddb1dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:18:26.477704 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 652/1303] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:18:26 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 04:18:26.745999 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f56716-4399-4631-b61a-15691f251707 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:26.848053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:26.856730 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f56716-4399-4631-b61a-15691f251707 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:26.895798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff8c73-1ec8-4f3f-8adc-b331a3ab3640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:26.896633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff8c73-1ec8-4f3f-8adc-b331a3ab3640 None admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:26.896886 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff8c73-1ec8-4f3f-8adc-b331a3ab3640 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:26.897153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff8c73-1ec8-4f3f-8adc-b331a3ab3640 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:26.897792 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff8c73-1ec8-4f3f-8adc-b331a3ab3640 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:26.899266 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff8c73-1ec8-4f3f-8adc-b331a3ab3640 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:26.909865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application [None req-1dff8c73-1ec8-4f3f-8adc-b331a3ab3640 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 04:18:26.909865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:18:26.909865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 04:18:26.909865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 04:18:26.909865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 04:18:26.909865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 04:18:26.909865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 04:18:26.909865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:18:26.909865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 04:18:26.909865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:18:26.909865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 04:18:26.909865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:18:26.909865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 04:18:26.909865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 04:18:26.909865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 04:18:26.909865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 04:18:26.909865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 04:18:26.910882 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 04:18:26.910882 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 04:18:26.910882 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 04:18:26.910882 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 04:18:26.910882 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 04:18:26.910882 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 04:18:26.910882 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 04:18:26.910882 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 04:18:26.910882 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 04:18:26.910882 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 04:18:26.910882 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 04:18:26.910882 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 04:18:26.910882 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 04:18:26.910882 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 04:18:26.910882 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 04:18:26.910882 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 04:18:26.912103 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 04:18:26.912103 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application with Lock( Aug 23 04:18:26.912103 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 04:18:26.912103 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self._enter() Aug 23 04:18:26.912103 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 04:18:26.912103 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 04:18:26.912103 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 04:18:26.912103 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application return self.creator() Aug 23 04:18:26.912103 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 04:18:26.912103 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application created_value = creator( Aug 23 04:18:26.912103 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 04:18:26.912103 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 04:18:26.912103 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 04:18:26.912103 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: ERROR keystone.server.flask.application Aug 23 04:18:26.912103 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 652/1304] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Aug 23 04:18:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 04:18:26.918127 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a496888-94d9-4245-9bf6-8b829ddb1dcf None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:27.009290 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:27.032790 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a496888-94d9-4245-9bf6-8b829ddb1dcf None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:27.067554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d78f0-fe5a-48fe-9a4d-082a6055c345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:27.068179 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d78f0-fe5a-48fe-9a4d-082a6055c345 None admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:27.068449 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d78f0-fe5a-48fe-9a4d-082a6055c345 None admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:27.068716 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d78f0-fe5a-48fe-9a4d-082a6055c345 None admin] PATH_INFO: `/v3/services` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:27.069401 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d78f0-fe5a-48fe-9a4d-082a6055c345 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:18:27.069658 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d78f0-fe5a-48fe-9a4d-082a6055c345 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:27.071024 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d78f0-fe5a-48fe-9a4d-082a6055c345 None admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:27.076229 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 653/1305] 10.222.0.18 () {56 vars in 1116 bytes} [Tue Aug 23 04:18:26 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:18:27.082421 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff8c73-1ec8-4f3f-8adc-b331a3ab3640 None admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:18:27.188993 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff8c73-1ec8-4f3f-8adc-b331a3ab3640 None admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:18:27.234994 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5c578-2be7-4dfd-a378-8192d1efcc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:18:27.235839 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5c578-2be7-4dfd-a378-8192d1efcc10 None admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:18:27.236247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5c578-2be7-4dfd-a378-8192d1efcc10 None admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:18:27.236605 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5c578-2be7-4dfd-a378-8192d1efcc10 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:18:27.237630 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e5c578-2be7-4dfd-a378-8192d1efcc10 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:18:27.239744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e5c578-2be7-4dfd-a378-8192d1efcc10 None admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:18:27.264003 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 653/1306] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:18:27 2022] POST /identity/v3/endpoints => generated 331 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:18:37.073549 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=30 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:18:37.259292 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:19:31.672067 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4379a6-b333-40f8-9cc4-a32026c9f191 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:31.672654 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4379a6-b333-40f8-9cc4-a32026c9f191 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:31.672654 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4379a6-b333-40f8-9cc4-a32026c9f191 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:31.674271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 654/1307] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:19:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:19:31.679861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a6ed5-a7d1-4f9e-baae-5e564af00f57 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:31.680352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a6ed5-a7d1-4f9e-baae-5e564af00f57 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:31.680439 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a6ed5-a7d1-4f9e-baae-5e564af00f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:31.767108 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-9c3a6ed5-a7d1-4f9e-baae-5e564af00f57 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:19:31.768363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-9c3a6ed5-a7d1-4f9e-baae-5e564af00f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:19:31.904334 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 654/1308] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:19:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:19:32.190520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d195d3f-d583-4963-953e-1a2cbc374727 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:32.190853 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d195d3f-d583-4963-953e-1a2cbc374727 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:32.191080 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d195d3f-d583-4963-953e-1a2cbc374727 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:32.192503 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 655/1309] 10.222.0.18 () {54 vars in 965 bytes} [Tue Aug 23 04:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:19:32.198708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4b539-43ea-4ef9-ae87-e650447432f8 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:32.199004 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4b539-43ea-4ef9-ae87-e650447432f8 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:32.199284 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4b539-43ea-4ef9-ae87-e650447432f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:32.285795 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-81b4b539-43ea-4ef9-ae87-e650447432f8 None None] MFA Rules not processed for user `fdca9c48e5d740f6bda8c48072b54c0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:19:32.286916 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-81b4b539-43ea-4ef9-ae87-e650447432f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:19:32.309424 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:19:32.379185 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 655/1310] 10.222.0.18 () {58 vars in 1062 bytes} [Tue Aug 23 04:19:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:19:32.380191 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d195d3f-d583-4963-953e-1a2cbc374727 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:19:32.381988 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-9d195d3f-d583-4963-953e-1a2cbc374727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:19:32.504287 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d195d3f-d583-4963-953e-1a2cbc374727 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:19:32.589377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8fac9e-a2d2-416e-a08c-141b257caa35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:19:32.590001 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8fac9e-a2d2-416e-a08c-141b257caa35 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:32.590275 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8fac9e-a2d2-416e-a08c-141b257caa35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:32.590541 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8fac9e-a2d2-416e-a08c-141b257caa35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:32.592254 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-bd8fac9e-a2d2-416e-a08c-141b257caa35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:19:32.651597 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:19:32.709703 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8fac9e-a2d2-416e-a08c-141b257caa35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:19:32.711623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8fac9e-a2d2-416e-a08c-141b257caa35 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:19:32.733527 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 656/1311] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:19:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:19:34.976940 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ebf07-6275-4667-a544-2a0f1094940b None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:34.977501 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ebf07-6275-4667-a544-2a0f1094940b None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:34.977501 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ebf07-6275-4667-a544-2a0f1094940b None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:34.978935 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 656/1312] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:19:34.985090 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc5848-cd41-4579-8ba9-f0c01c5b44c8 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:34.985653 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc5848-cd41-4579-8ba9-f0c01c5b44c8 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:34.985653 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc5848-cd41-4579-8ba9-f0c01c5b44c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:35.066547 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-b8fc5848-cd41-4579-8ba9-f0c01c5b44c8 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:19:35.067770 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-b8fc5848-cd41-4579-8ba9-f0c01c5b44c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:19:35.169210 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 657/1313] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:19:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:19:35.470370 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb7c8a-f2f9-44fa-89bc-00173e56547a None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:35.470677 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb7c8a-f2f9-44fa-89bc-00173e56547a None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:35.470991 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb7c8a-f2f9-44fa-89bc-00173e56547a None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:35.473149 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 657/1314] 10.222.0.18 () {54 vars in 965 bytes} [Tue Aug 23 04:19:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:19:35.479738 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284effe-b88a-46bc-9b64-02c7005819d4 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:35.480065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284effe-b88a-46bc-9b64-02c7005819d4 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:35.480345 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284effe-b88a-46bc-9b64-02c7005819d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:35.493072 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:19:35.559830 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-e284effe-b88a-46bc-9b64-02c7005819d4 None None] MFA Rules not processed for user `fdca9c48e5d740f6bda8c48072b54c0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:19:35.561062 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-e284effe-b88a-46bc-9b64-02c7005819d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:19:35.642151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 658/1315] 10.222.0.18 () {58 vars in 1062 bytes} [Tue Aug 23 04:19:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:19:35.652446 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb7c8a-f2f9-44fa-89bc-00173e56547a None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:19:35.654417 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-74bb7c8a-f2f9-44fa-89bc-00173e56547a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:19:35.777691 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb7c8a-f2f9-44fa-89bc-00173e56547a None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:19:35.850168 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:19:35.853146 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f4702-baf6-4d33-85bd-1aa3d7e6af0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:19:35.853757 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f4702-baf6-4d33-85bd-1aa3d7e6af0e service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:35.854036 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f4702-baf6-4d33-85bd-1aa3d7e6af0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:35.854353 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f4702-baf6-4d33-85bd-1aa3d7e6af0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:35.856438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-d38f4702-baf6-4d33-85bd-1aa3d7e6af0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:19:35.973081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38f4702-baf6-4d33-85bd-1aa3d7e6af0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:19:35.974533 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38f4702-baf6-4d33-85bd-1aa3d7e6af0e service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:19:36.000937 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 658/1316] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:19:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:19:38.431168 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c7e5b-6f30-428c-8111-c330327a9f4a None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:38.431789 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c7e5b-6f30-428c-8111-c330327a9f4a None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:38.431789 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c7e5b-6f30-428c-8111-c330327a9f4a None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:38.433093 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 659/1317] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:19:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:19:38.439610 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987da1e-be79-4f59-9327-37ceb8591f8f None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:38.440345 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987da1e-be79-4f59-9327-37ceb8591f8f None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:38.440527 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987da1e-be79-4f59-9327-37ceb8591f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:38.520810 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-1987da1e-be79-4f59-9327-37ceb8591f8f None None] MFA Rules not processed for user `628f03c8b9b44e27bacc6919029346d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:19:38.522045 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-1987da1e-be79-4f59-9327-37ceb8591f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:19:38.576725 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:19:38.627334 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 659/1318] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:19:38 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:19:38.908407 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c7e5b-6f30-428c-8111-c330327a9f4a None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:19:38.910844 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-7a5c7e5b-6f30-428c-8111-c330327a9f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:19:39.030722 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c7e5b-6f30-428c-8111-c330327a9f4a None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:19:39.033534 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:19:39.113607 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fd3a7-49f1-4cbc-b1e5-a5521edacf0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:19:39.114213 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fd3a7-49f1-4cbc-b1e5-a5521edacf0c service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:39.114495 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fd3a7-49f1-4cbc-b1e5-a5521edacf0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:39.114761 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fd3a7-49f1-4cbc-b1e5-a5521edacf0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:39.116396 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-656fd3a7-49f1-4cbc-b1e5-a5521edacf0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:19:39.239229 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656fd3a7-49f1-4cbc-b1e5-a5521edacf0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:19:39.240627 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656fd3a7-49f1-4cbc-b1e5-a5521edacf0c service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:19:39.259687 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 660/1319] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:19:38 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:19:41.732148 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a094fd6-172b-48f0-a134-fceb12c71edf None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:41.733081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a094fd6-172b-48f0-a134-fceb12c71edf None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:41.733081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a094fd6-172b-48f0-a134-fceb12c71edf None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:41.735181 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 660/1320] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:19:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:19:41.744867 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8497905-d27f-42e3-bbdc-494afa516039 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:41.745291 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8497905-d27f-42e3-bbdc-494afa516039 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:41.745464 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8497905-d27f-42e3-bbdc-494afa516039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:41.794666 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:19:41.822896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-e8497905-d27f-42e3-bbdc-494afa516039 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:19:41.824144 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-e8497905-d27f-42e3-bbdc-494afa516039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:19:41.934309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 661/1321] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:19:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:19:42.220411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a094fd6-172b-48f0-a134-fceb12c71edf None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:19:42.222880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-1a094fd6-172b-48f0-a134-fceb12c71edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:19:42.336680 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a094fd6-172b-48f0-a134-fceb12c71edf None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:19:42.420889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb805821-af90-4999-be27-300ca71e6da5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:19:42.421827 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb805821-af90-4999-be27-300ca71e6da5 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:42.422284 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb805821-af90-4999-be27-300ca71e6da5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:42.422726 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb805821-af90-4999-be27-300ca71e6da5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:42.425383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-cb805821-af90-4999-be27-300ca71e6da5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:19:42.438260 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:19:42.546432 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb805821-af90-4999-be27-300ca71e6da5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:19:42.547885 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb805821-af90-4999-be27-300ca71e6da5 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:19:42.569623 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 661/1322] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:19:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:19:45.164155 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89079bed-a726-48d5-bb43-5e8f03de80ad None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:45.164771 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89079bed-a726-48d5-bb43-5e8f03de80ad None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:45.164771 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89079bed-a726-48d5-bb43-5e8f03de80ad None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:45.166810 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 662/1323] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:19:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:19:45.174636 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af965c4-4be1-4a3f-9ce2-2b3bc8053502 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:45.175111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af965c4-4be1-4a3f-9ce2-2b3bc8053502 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:45.175304 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af965c4-4be1-4a3f-9ce2-2b3bc8053502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:45.244906 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-2af965c4-4be1-4a3f-9ce2-2b3bc8053502 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:19:45.246100 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-2af965c4-4be1-4a3f-9ce2-2b3bc8053502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:19:45.305501 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:19:45.347255 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 662/1324] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:19:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:19:45.645162 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89079bed-a726-48d5-bb43-5e8f03de80ad None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:19:45.646787 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-89079bed-a726-48d5-bb43-5e8f03de80ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:19:45.757191 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89079bed-a726-48d5-bb43-5e8f03de80ad None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:19:45.770158 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:19:45.840191 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50769f1d-9fe7-4266-96b6-524a5423420e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:19:45.840701 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50769f1d-9fe7-4266-96b6-524a5423420e service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:45.840976 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50769f1d-9fe7-4266-96b6-524a5423420e service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:45.841246 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50769f1d-9fe7-4266-96b6-524a5423420e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:45.842853 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-50769f1d-9fe7-4266-96b6-524a5423420e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:19:45.961589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50769f1d-9fe7-4266-96b6-524a5423420e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:19:45.963040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50769f1d-9fe7-4266-96b6-524a5423420e service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:19:45.981556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 663/1325] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:19:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:19:47.962326 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5eb55f-3534-47cd-b280-cd076d8670c2 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:47.962921 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5eb55f-3534-47cd-b280-cd076d8670c2 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:47.962921 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5eb55f-3534-47cd-b280-cd076d8670c2 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:47.964847 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 663/1326] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:19:47.970140 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a10bc2-a75e-45a3-95e9-b39fcf53f923 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:47.970598 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a10bc2-a75e-45a3-95e9-b39fcf53f923 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:47.970699 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a10bc2-a75e-45a3-95e9-b39fcf53f923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:48.020125 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:19:48.039710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-a4a10bc2-a75e-45a3-95e9-b39fcf53f923 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:19:48.040955 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-a4a10bc2-a75e-45a3-95e9-b39fcf53f923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:19:48.140356 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 664/1327] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:19:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:19:48.420966 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5eb55f-3534-47cd-b280-cd076d8670c2 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:19:48.422506 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-6a5eb55f-3534-47cd-b280-cd076d8670c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:19:48.540226 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5eb55f-3534-47cd-b280-cd076d8670c2 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:19:48.618886 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e07454-7945-4b2d-b7dc-4aa564410522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:19:48.619493 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e07454-7945-4b2d-b7dc-4aa564410522 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:48.619766 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e07454-7945-4b2d-b7dc-4aa564410522 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:48.620084 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e07454-7945-4b2d-b7dc-4aa564410522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:48.621641 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-58e07454-7945-4b2d-b7dc-4aa564410522 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:19:48.638941 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:19:48.737201 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e07454-7945-4b2d-b7dc-4aa564410522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:19:48.738551 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e07454-7945-4b2d-b7dc-4aa564410522 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:19:48.758913 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 664/1328] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:19:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:19:50.717343 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b3b5e-5074-41a7-9bba-0efc135b8336 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:50.717978 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b3b5e-5074-41a7-9bba-0efc135b8336 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:50.717978 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b3b5e-5074-41a7-9bba-0efc135b8336 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:50.719248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 665/1329] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:19:50.725299 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3b1d4-33d6-4bea-a119-c9d7953aa7c1 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:50.725840 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3b1d4-33d6-4bea-a119-c9d7953aa7c1 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:50.725840 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3b1d4-33d6-4bea-a119-c9d7953aa7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:50.792835 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-19c3b1d4-33d6-4bea-a119-c9d7953aa7c1 None None] MFA Rules not processed for user `628f03c8b9b44e27bacc6919029346d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:19:50.794202 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-19c3b1d4-33d6-4bea-a119-c9d7953aa7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:19:50.860074 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:19:50.893875 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 665/1330] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:19:50 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:19:51.181007 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b3b5e-5074-41a7-9bba-0efc135b8336 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:19:51.182581 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-3c1b3b5e-5074-41a7-9bba-0efc135b8336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:19:51.291317 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b3b5e-5074-41a7-9bba-0efc135b8336 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:19:51.312442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:19:51.372512 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba5f8e-cb05-4c53-aaa4-4626c0b359b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:19:51.373134 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba5f8e-cb05-4c53-aaa4-4626c0b359b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:51.373420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba5f8e-cb05-4c53-aaa4-4626c0b359b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:51.373685 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba5f8e-cb05-4c53-aaa4-4626c0b359b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:51.375274 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-9cba5f8e-cb05-4c53-aaa4-4626c0b359b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:19:51.497290 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba5f8e-cb05-4c53-aaa4-4626c0b359b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:19:51.498766 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba5f8e-cb05-4c53-aaa4-4626c0b359b3 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:19:51.521564 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 666/1331] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:19:51 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:19:53.876298 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2903c-39c1-4641-a84a-679edd3e72e3 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:53.876898 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2903c-39c1-4641-a84a-679edd3e72e3 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:53.876898 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2903c-39c1-4641-a84a-679edd3e72e3 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:53.878226 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 666/1332] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:19:53.884640 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa15d2-6130-45b7-bc0e-27fe953c6b02 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:53.885323 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa15d2-6130-45b7-bc0e-27fe953c6b02 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:53.885438 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa15d2-6130-45b7-bc0e-27fe953c6b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:53.957182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:19:53.960505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-d1fa15d2-6130-45b7-bc0e-27fe953c6b02 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:19:53.962305 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-d1fa15d2-6130-45b7-bc0e-27fe953c6b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:19:54.067503 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 667/1333] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:19:54.358503 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a2903c-39c1-4641-a84a-679edd3e72e3 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:19:54.361073 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-b3a2903c-39c1-4641-a84a-679edd3e72e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:19:54.486130 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a2903c-39c1-4641-a84a-679edd3e72e3 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:19:54.562309 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668f49d3-0f0e-4524-a75a-57e19736392b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:19:54.562945 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f49d3-0f0e-4524-a75a-57e19736392b service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:54.563228 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f49d3-0f0e-4524-a75a-57e19736392b service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:54.563486 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f49d3-0f0e-4524-a75a-57e19736392b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:54.565384 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-668f49d3-0f0e-4524-a75a-57e19736392b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:19:54.595554 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:19:54.682118 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668f49d3-0f0e-4524-a75a-57e19736392b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:19:54.683569 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668f49d3-0f0e-4524-a75a-57e19736392b service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:19:54.702064 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 667/1334] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:19:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:19:56.640075 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18588909-ce6a-4a76-a07a-eeda9215c48e None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:56.640639 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18588909-ce6a-4a76-a07a-eeda9215c48e None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:56.640639 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18588909-ce6a-4a76-a07a-eeda9215c48e None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:56.641965 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 668/1335] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:19:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:19:56.648418 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151232b-7f9b-4af3-b3af-c659bc572684 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:56.649125 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151232b-7f9b-4af3-b3af-c659bc572684 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:56.649322 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151232b-7f9b-4af3-b3af-c659bc572684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:56.725340 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-9151232b-7f9b-4af3-b3af-c659bc572684 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:19:56.726495 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-9151232b-7f9b-4af3-b3af-c659bc572684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:19:56.806949 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:19:56.834886 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 668/1336] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:19:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:19:57.130870 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18588909-ce6a-4a76-a07a-eeda9215c48e None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:19:57.132493 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-18588909-ce6a-4a76-a07a-eeda9215c48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:19:57.247133 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18588909-ce6a-4a76-a07a-eeda9215c48e None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:19:57.276405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:19:57.335496 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ca715-c3fc-4429-bc09-19669ea3fa9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:19:57.336690 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ca715-c3fc-4429-bc09-19669ea3fa9f service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:57.336690 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ca715-c3fc-4429-bc09-19669ea3fa9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:57.336690 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ca715-c3fc-4429-bc09-19669ea3fa9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:57.338290 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-3c0ca715-c3fc-4429-bc09-19669ea3fa9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:19:57.454730 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ca715-c3fc-4429-bc09-19669ea3fa9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:19:57.454730 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ca715-c3fc-4429-bc09-19669ea3fa9f service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:19:57.482705 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 669/1337] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:19:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:19:59.841151 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b0cd1-fd1f-44bd-8ec6-43b8bb9d7192 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:59.842073 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b0cd1-fd1f-44bd-8ec6-43b8bb9d7192 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:59.842073 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b0cd1-fd1f-44bd-8ec6-43b8bb9d7192 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:59.844206 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 669/1338] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:19:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:19:59.852198 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a3867-f65f-4977-b311-4c0ce08e25ba None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:19:59.852670 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a3867-f65f-4977-b311-4c0ce08e25ba None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:19:59.852783 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a3867-f65f-4977-b311-4c0ce08e25ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:19:59.924366 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-fd0a3867-f65f-4977-b311-4c0ce08e25ba None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:19:59.925589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-fd0a3867-f65f-4977-b311-4c0ce08e25ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:19:59.934248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:00.025862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 670/1339] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:19:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:20:00.312506 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b0cd1-fd1f-44bd-8ec6-43b8bb9d7192 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:20:00.314963 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-3b0b0cd1-fd1f-44bd-8ec6-43b8bb9d7192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:00.440219 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b0cd1-fd1f-44bd-8ec6-43b8bb9d7192 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:20:00.521560 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226fc75e-f3b0-47f5-9a89-836e248b6e4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:20:00.522176 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fc75e-f3b0-47f5-9a89-836e248b6e4c service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:00.522452 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fc75e-f3b0-47f5-9a89-836e248b6e4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:00.522729 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fc75e-f3b0-47f5-9a89-836e248b6e4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:00.524347 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-226fc75e-f3b0-47f5-9a89-836e248b6e4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:00.559945 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:00.649124 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226fc75e-f3b0-47f5-9a89-836e248b6e4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:20:00.650626 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226fc75e-f3b0-47f5-9a89-836e248b6e4c service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:20:00.669117 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 670/1340] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:20:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:20:04.646594 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa033c40-23ec-4114-870f-3c00df42f34b None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:04.647169 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa033c40-23ec-4114-870f-3c00df42f34b None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:04.647169 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa033c40-23ec-4114-870f-3c00df42f34b None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:04.648502 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 671/1341] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:20:04.654503 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec18c9c8-8607-42f8-904a-bb2ae29b6bbe None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:04.655017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec18c9c8-8607-42f8-904a-bb2ae29b6bbe None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:04.655121 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec18c9c8-8607-42f8-904a-bb2ae29b6bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:04.724948 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-ec18c9c8-8607-42f8-904a-bb2ae29b6bbe None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:20:04.726172 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-ec18c9c8-8607-42f8-904a-bb2ae29b6bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:04.820223 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:04.833334 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 671/1342] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:20:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:20:05.124548 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa033c40-23ec-4114-870f-3c00df42f34b None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:20:05.126192 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-fa033c40-23ec-4114-870f-3c00df42f34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:05.237591 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa033c40-23ec-4114-870f-3c00df42f34b None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:20:05.278605 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:05.315653 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b8d550-6940-4a1c-ab4e-7ca1a1628e44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:20:05.316300 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b8d550-6940-4a1c-ab4e-7ca1a1628e44 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:05.316555 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b8d550-6940-4a1c-ab4e-7ca1a1628e44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:05.316838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b8d550-6940-4a1c-ab4e-7ca1a1628e44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:05.318417 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-20b8d550-6940-4a1c-ab4e-7ca1a1628e44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:05.432032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b8d550-6940-4a1c-ab4e-7ca1a1628e44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:20:05.433500 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b8d550-6940-4a1c-ab4e-7ca1a1628e44 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:20:05.452231 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 672/1343] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:20:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:20:07.982317 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a8e24-d750-4b95-9de9-06c012ddb8ff None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:07.982823 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a8e24-d750-4b95-9de9-06c012ddb8ff None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:07.982918 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a8e24-d750-4b95-9de9-06c012ddb8ff None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:07.984358 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 672/1344] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:20:07.989609 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d86136-7e74-4e88-8ca4-61de226a4dd9 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:07.990088 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d86136-7e74-4e88-8ca4-61de226a4dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:07.990183 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d86136-7e74-4e88-8ca4-61de226a4dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:08.062922 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-a2d86136-7e74-4e88-8ca4-61de226a4dd9 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:20:08.064171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-a2d86136-7e74-4e88-8ca4-61de226a4dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:08.083462 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:08.168604 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 673/1345] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:20:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:20:08.449184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a8e24-d750-4b95-9de9-06c012ddb8ff None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:20:08.450734 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-0b4a8e24-d750-4b95-9de9-06c012ddb8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:08.557692 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a8e24-d750-4b95-9de9-06c012ddb8ff None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:20:08.636652 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca11e1a-9cd9-45f6-8d0b-7b109b706ad3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:20:08.637284 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca11e1a-9cd9-45f6-8d0b-7b109b706ad3 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:08.637541 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca11e1a-9cd9-45f6-8d0b-7b109b706ad3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:08.637894 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca11e1a-9cd9-45f6-8d0b-7b109b706ad3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:08.639607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-dca11e1a-9cd9-45f6-8d0b-7b109b706ad3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:08.686555 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:08.757455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca11e1a-9cd9-45f6-8d0b-7b109b706ad3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:20:08.758813 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca11e1a-9cd9-45f6-8d0b-7b109b706ad3 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:20:08.779020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 673/1346] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:20:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:20:12.669625 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31400c-0883-436b-bd29-a187a2c3b194 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:12.670177 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31400c-0883-436b-bd29-a187a2c3b194 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:12.670177 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31400c-0883-436b-bd29-a187a2c3b194 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:12.672219 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 674/1347] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:20:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:20:12.677774 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c57a7c-6557-4195-96b7-130f1174b4ef None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:12.678220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c57a7c-6557-4195-96b7-130f1174b4ef None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:12.678310 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c57a7c-6557-4195-96b7-130f1174b4ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:12.763670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-b9c57a7c-6557-4195-96b7-130f1174b4ef None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:20:12.767207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-b9c57a7c-6557-4195-96b7-130f1174b4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:12.874305 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:12.884091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 674/1348] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:20:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:20:13.160212 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31400c-0883-436b-bd29-a187a2c3b194 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:20:13.161795 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-bc31400c-0883-436b-bd29-a187a2c3b194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:13.268861 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31400c-0883-436b-bd29-a187a2c3b194 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:20:13.317324 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:13.347467 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5168e927-bb4b-43ef-b5b8-cca0a111adff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:20:13.348380 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168e927-bb4b-43ef-b5b8-cca0a111adff service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:13.348380 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168e927-bb4b-43ef-b5b8-cca0a111adff service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:13.348633 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168e927-bb4b-43ef-b5b8-cca0a111adff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:13.350472 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-5168e927-bb4b-43ef-b5b8-cca0a111adff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:13.482445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5168e927-bb4b-43ef-b5b8-cca0a111adff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:20:13.483961 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5168e927-bb4b-43ef-b5b8-cca0a111adff service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:20:13.502565 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 675/1349] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:20:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:20:18.482444 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e2b6d-3b52-477a-8c33-4242b33eb4e8 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:18.483421 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e2b6d-3b52-477a-8c33-4242b33eb4e8 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:18.483899 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e2b6d-3b52-477a-8c33-4242b33eb4e8 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:18.486202 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 675/1350] 10.222.0.18 () {54 vars in 921 bytes} [Tue Aug 23 04:20:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:20:18.491542 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21852c16-972e-4f98-aea6-07c1b0aa5abf None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:18.492126 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21852c16-972e-4f98-aea6-07c1b0aa5abf None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:18.492126 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21852c16-972e-4f98-aea6-07c1b0aa5abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:18.599196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-21852c16-972e-4f98-aea6-07c1b0aa5abf None None] MFA Rules not processed for user `dd503fe3cf6d428e962c14ab864ee608`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:20:18.600562 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-21852c16-972e-4f98-aea6-07c1b0aa5abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:18.620368 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:18.739604 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 676/1351] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Aug 23 04:20:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:20:22.946527 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b78d57-b443-415a-939a-92c9c93bf31b None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:22.947683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b78d57-b443-415a-939a-92c9c93bf31b None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:22.948012 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b78d57-b443-415a-939a-92c9c93bf31b None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:22.949531 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 676/1352] 10.222.0.18 () {54 vars in 921 bytes} [Tue Aug 23 04:20:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:20:22.954585 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820df8f6-b929-4647-ab6f-009197220b1a None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:22.954998 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820df8f6-b929-4647-ab6f-009197220b1a None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:22.955137 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820df8f6-b929-4647-ab6f-009197220b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:23.038570 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-820df8f6-b929-4647-ab6f-009197220b1a None None] MFA Rules not processed for user `dd503fe3cf6d428e962c14ab864ee608`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:20:23.040056 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-820df8f6-b929-4647-ab6f-009197220b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:23.125902 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:23.167413 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 677/1353] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Aug 23 04:20:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:20:24.597873 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117510f1-535a-4347-b753-1a422489135f None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:24.598413 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117510f1-535a-4347-b753-1a422489135f None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:24.598413 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117510f1-535a-4347-b753-1a422489135f None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:24.599985 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 677/1354] 10.222.0.18 () {54 vars in 921 bytes} [Tue Aug 23 04:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:20:24.605258 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c4956-f446-4691-bb07-c38f3ec89963 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:24.605675 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c4956-f446-4691-bb07-c38f3ec89963 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:24.605966 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c4956-f446-4691-bb07-c38f3ec89963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:24.684987 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-073c4956-f446-4691-bb07-c38f3ec89963 None None] MFA Rules not processed for user `dd503fe3cf6d428e962c14ab864ee608`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:20:24.686573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-073c4956-f446-4691-bb07-c38f3ec89963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:24.798670 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 678/1355] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Aug 23 04:20:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:20:28.065553 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ffcaa-fb21-4b62-8d0d-eab7c50714f2 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:28.066153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ffcaa-fb21-4b62-8d0d-eab7c50714f2 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:28.066153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ffcaa-fb21-4b62-8d0d-eab7c50714f2 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:28.067465 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 678/1356] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:20:28.073044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8fd1c-ccb3-40aa-a791-d5441e214f01 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:28.073470 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8fd1c-ccb3-40aa-a791-d5441e214f01 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:28.073737 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8fd1c-ccb3-40aa-a791-d5441e214f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:28.116053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:28.154009 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-aad8fd1c-ccb3-40aa-a791-d5441e214f01 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:20:28.155217 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-aad8fd1c-ccb3-40aa-a791-d5441e214f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:28.298192 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 679/1357] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:20:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:20:28.597564 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57f43a-3145-410e-a7ad-1e404b6a3d9c None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:28.598067 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57f43a-3145-410e-a7ad-1e404b6a3d9c None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:28.598343 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57f43a-3145-410e-a7ad-1e404b6a3d9c None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:28.599696 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 679/1358] 10.222.0.18 () {54 vars in 962 bytes} [Tue Aug 23 04:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:20:28.606486 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b09840-c1d8-41a1-a792-29581b110bf2 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:28.606777 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b09840-c1d8-41a1-a792-29581b110bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:28.607054 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b09840-c1d8-41a1-a792-29581b110bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:28.727540 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-c4b09840-c1d8-41a1-a792-29581b110bf2 None None] MFA Rules not processed for user `cd7b11f479a54adc8d9b8641859b07a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:20:28.729167 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-c4b09840-c1d8-41a1-a792-29581b110bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:28.791019 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:28.867868 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 680/1359] 10.222.0.18 () {58 vars in 1059 bytes} [Tue Aug 23 04:20:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:20:28.876330 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b57f43a-3145-410e-a7ad-1e404b6a3d9c None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:20:28.878051 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-1b57f43a-3145-410e-a7ad-1e404b6a3d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:29.053422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b57f43a-3145-410e-a7ad-1e404b6a3d9c None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:20:29.133590 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:29.183772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc20b4-5744-4c24-bf84-21aa8b9ae31a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:20:29.184491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc20b4-5744-4c24-bf84-21aa8b9ae31a service nova] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:29.184781 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc20b4-5744-4c24-bf84-21aa8b9ae31a service nova] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:29.185047 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc20b4-5744-4c24-bf84-21aa8b9ae31a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:29.186908 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-9ddc20b4-5744-4c24-bf84-21aa8b9ae31a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:29.283481 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddc20b4-5744-4c24-bf84-21aa8b9ae31a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:20:29.285137 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddc20b4-5744-4c24-bf84-21aa8b9ae31a service nova] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:20:29.303860 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 680/1360] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:20:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:20:31.426383 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744b524-1351-4a28-a9bc-5e9cea275a71 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:31.426929 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744b524-1351-4a28-a9bc-5e9cea275a71 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:31.426929 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744b524-1351-4a28-a9bc-5e9cea275a71 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:31.428430 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 681/1361] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:20:31.433978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9e5ea-656d-4588-bb76-11e434f9a50c None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:31.434430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9e5ea-656d-4588-bb76-11e434f9a50c None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:31.434529 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9e5ea-656d-4588-bb76-11e434f9a50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:31.515990 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-73c9e5ea-656d-4588-bb76-11e434f9a50c None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:20:31.517384 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-73c9e5ea-656d-4588-bb76-11e434f9a50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:31.558560 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:31.631472 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 681/1362] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:20:31.922210 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f744b524-1351-4a28-a9bc-5e9cea275a71 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:20:31.923741 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-f744b524-1351-4a28-a9bc-5e9cea275a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:32.083402 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f744b524-1351-4a28-a9bc-5e9cea275a71 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:20:32.093687 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:32.220821 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc24408-37c5-4a87-8d5a-13befd72ab83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:20:32.221542 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc24408-37c5-4a87-8d5a-13befd72ab83 service nova] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:32.221938 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc24408-37c5-4a87-8d5a-13befd72ab83 service nova] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:32.222438 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc24408-37c5-4a87-8d5a-13befd72ab83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:32.224454 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-3fc24408-37c5-4a87-8d5a-13befd72ab83 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:32.331237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc24408-37c5-4a87-8d5a-13befd72ab83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:20:32.333115 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc24408-37c5-4a87-8d5a-13befd72ab83 service nova] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:20:32.355642 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 682/1363] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:20:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:20:34.596242 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ba6a3-9caa-41f8-9ecf-4ca7f74208a8 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:34.596777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ba6a3-9caa-41f8-9ecf-4ca7f74208a8 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:34.596777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ba6a3-9caa-41f8-9ecf-4ca7f74208a8 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:34.598297 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 682/1364] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:20:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:20:34.603815 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bebaf7-e2f0-448d-9263-7ae217aa104f None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:34.604739 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bebaf7-e2f0-448d-9263-7ae217aa104f None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:34.605229 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bebaf7-e2f0-448d-9263-7ae217aa104f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:34.622605 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:34.678671 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-d7bebaf7-e2f0-448d-9263-7ae217aa104f None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:20:34.679867 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-d7bebaf7-e2f0-448d-9263-7ae217aa104f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:34.792072 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 683/1365] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:20:35.081927 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14924844-6fd6-475b-b82e-5dbcb389c815 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:35.082192 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14924844-6fd6-475b-b82e-5dbcb389c815 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:35.082644 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14924844-6fd6-475b-b82e-5dbcb389c815 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:35.084009 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 683/1366] 10.222.0.18 () {54 vars in 962 bytes} [Tue Aug 23 04:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:20:35.089909 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a89e414-f145-472d-bba1-1770259c78e0 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:35.090522 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a89e414-f145-472d-bba1-1770259c78e0 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:35.090955 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a89e414-f145-472d-bba1-1770259c78e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:35.174715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-8a89e414-f145-472d-bba1-1770259c78e0 None None] MFA Rules not processed for user `cd7b11f479a54adc8d9b8641859b07a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:20:35.176138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-8a89e414-f145-472d-bba1-1770259c78e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:35.212695 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:35.302242 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 684/1367] 10.222.0.18 () {58 vars in 1059 bytes} [Tue Aug 23 04:20:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:20:35.310840 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14924844-6fd6-475b-b82e-5dbcb389c815 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:20:35.312844 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-14924844-6fd6-475b-b82e-5dbcb389c815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:35.494882 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14924844-6fd6-475b-b82e-5dbcb389c815 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:20:35.520522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:35.627620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5ad2b-afac-43c5-b81a-8e86ffcaadf7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:20:35.628311 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5ad2b-afac-43c5-b81a-8e86ffcaadf7 service nova] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:35.628573 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5ad2b-afac-43c5-b81a-8e86ffcaadf7 service nova] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:35.628842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5ad2b-afac-43c5-b81a-8e86ffcaadf7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:35.630529 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-57d5ad2b-afac-43c5-b81a-8e86ffcaadf7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:35.739288 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5ad2b-afac-43c5-b81a-8e86ffcaadf7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:20:35.741313 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5ad2b-afac-43c5-b81a-8e86ffcaadf7 service nova] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:20:35.761934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 684/1368] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:20:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:20:36.299393 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9b736-d799-4bb3-b40c-31864158b69f None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:36.299986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9b736-d799-4bb3-b40c-31864158b69f None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:36.300284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9b736-d799-4bb3-b40c-31864158b69f None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:36.301565 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 685/1369] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:20:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:20:36.307956 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04aed19-54ba-4604-a43c-8f41b3a4e9ff None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:36.308260 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04aed19-54ba-4604-a43c-8f41b3a4e9ff None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:36.308678 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04aed19-54ba-4604-a43c-8f41b3a4e9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:36.343696 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:36.380979 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-b04aed19-54ba-4604-a43c-8f41b3a4e9ff None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:20:36.382180 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-b04aed19-54ba-4604-a43c-8f41b3a4e9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:36.479960 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 685/1370] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:20:37.058100 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7acf3-dc61-4072-841e-b4bde370fcea None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:37.058444 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7acf3-dc61-4072-841e-b4bde370fcea None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:37.058776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7acf3-dc61-4072-841e-b4bde370fcea None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:37.060459 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 686/1371] 10.222.0.18 () {54 vars in 964 bytes} [Tue Aug 23 04:20:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:20:37.065380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eff95d-6b49-4349-8b1b-69609ecc6ad9 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:37.065854 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eff95d-6b49-4349-8b1b-69609ecc6ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:37.065936 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eff95d-6b49-4349-8b1b-69609ecc6ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:37.147974 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-62eff95d-6b49-4349-8b1b-69609ecc6ad9 None None] MFA Rules not processed for user `2db57fe3915446acb57b4f84ca353379`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:20:37.149208 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-62eff95d-6b49-4349-8b1b-69609ecc6ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:37.204625 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:37.240611 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 686/1372] 10.222.0.18 () {58 vars in 1061 bytes} [Tue Aug 23 04:20:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:20:37.250840 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7acf3-dc61-4072-841e-b4bde370fcea None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:20:37.252558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-0fd7acf3-dc61-4072-841e-b4bde370fcea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:37.363588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7acf3-dc61-4072-841e-b4bde370fcea None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:20:37.406683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:37.444986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6f06d-a79b-4e05-9377-e27a4b22a88e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:20:37.445588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6f06d-a79b-4e05-9377-e27a4b22a88e service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:37.445995 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6f06d-a79b-4e05-9377-e27a4b22a88e service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:37.446269 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6f06d-a79b-4e05-9377-e27a4b22a88e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:37.448484 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-8aa6f06d-a79b-4e05-9377-e27a4b22a88e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:37.609307 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa6f06d-a79b-4e05-9377-e27a4b22a88e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:20:37.610768 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa6f06d-a79b-4e05-9377-e27a4b22a88e service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:20:37.633273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 687/1373] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:20:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:20:37.784491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71749a-d321-4df4-9211-77b26939b18f None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:37.786431 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71749a-d321-4df4-9211-77b26939b18f None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:37.786431 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71749a-d321-4df4-9211-77b26939b18f None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:37.786431 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 687/1374] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:20:37.791870 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de41fc8-2ca1-469e-abe2-1d28bdea6101 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:37.792126 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de41fc8-2ca1-469e-abe2-1d28bdea6101 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:37.792386 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de41fc8-2ca1-469e-abe2-1d28bdea6101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:37.864319 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-2de41fc8-2ca1-469e-abe2-1d28bdea6101 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:20:37.865472 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-2de41fc8-2ca1-469e-abe2-1d28bdea6101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:37.886541 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:37.971121 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 688/1375] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:20:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:20:38.263100 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71749a-d321-4df4-9211-77b26939b18f None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:20:38.264712 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-dd71749a-d321-4df4-9211-77b26939b18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:38.420241 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71749a-d321-4df4-9211-77b26939b18f None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:20:38.480752 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:38.544401 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec0346-bd94-4018-9409-c83030fea8ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:20:38.544933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec0346-bd94-4018-9409-c83030fea8ca service nova] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:38.545212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec0346-bd94-4018-9409-c83030fea8ca service nova] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:38.545479 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec0346-bd94-4018-9409-c83030fea8ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:38.547102 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a0ec0346-bd94-4018-9409-c83030fea8ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:38.641353 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec0346-bd94-4018-9409-c83030fea8ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:20:38.642967 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec0346-bd94-4018-9409-c83030fea8ca service nova] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:20:38.661379 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 688/1376] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:20:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:20:40.608405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f6ddc-7ecf-4385-921e-a40bcf64c86d None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:40.608883 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f6ddc-7ecf-4385-921e-a40bcf64c86d None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:40.609008 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f6ddc-7ecf-4385-921e-a40bcf64c86d None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:40.610308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 689/1377] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:20:40.615741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ce588-d59d-42de-9eba-e21a29f00cc4 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:40.616197 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ce588-d59d-42de-9eba-e21a29f00cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:40.616309 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ce588-d59d-42de-9eba-e21a29f00cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:40.695073 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-182ce588-d59d-42de-9eba-e21a29f00cc4 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:20:40.696390 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-182ce588-d59d-42de-9eba-e21a29f00cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:40.705875 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:40.801795 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 689/1378] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:20:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:20:41.088419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f6ddc-7ecf-4385-921e-a40bcf64c86d None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:20:41.090157 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-596f6ddc-7ecf-4385-921e-a40bcf64c86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:41.301602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f6ddc-7ecf-4385-921e-a40bcf64c86d None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:20:41.305670 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:41.422917 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4018f53-85a0-4c52-9454-bcdb6cfd69d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:20:41.423526 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4018f53-85a0-4c52-9454-bcdb6cfd69d7 service nova] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:41.423827 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4018f53-85a0-4c52-9454-bcdb6cfd69d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:41.424125 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4018f53-85a0-4c52-9454-bcdb6cfd69d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:41.425696 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-a4018f53-85a0-4c52-9454-bcdb6cfd69d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:41.522250 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4018f53-85a0-4c52-9454-bcdb6cfd69d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:20:41.523869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4018f53-85a0-4c52-9454-bcdb6cfd69d7 service nova] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:20:41.542390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 690/1379] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:20:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:20:43.478782 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233d8f3-5f9c-4cdf-9ec9-9c4dcbde097a None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:43.479231 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233d8f3-5f9c-4cdf-9ec9-9c4dcbde097a None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:43.479532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233d8f3-5f9c-4cdf-9ec9-9c4dcbde097a None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:43.480817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 690/1380] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:20:43.486331 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17711d8e-dbe5-4dad-ac0d-63f9ee994e2f None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:43.486734 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17711d8e-dbe5-4dad-ac0d-63f9ee994e2f None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:43.486885 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17711d8e-dbe5-4dad-ac0d-63f9ee994e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:43.498834 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:43.558898 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-17711d8e-dbe5-4dad-ac0d-63f9ee994e2f None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:20:43.560378 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-17711d8e-dbe5-4dad-ac0d-63f9ee994e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:43.667169 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 691/1381] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:20:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:20:43.956643 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233d8f3-5f9c-4cdf-9ec9-9c4dcbde097a None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:20:43.958300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-0233d8f3-5f9c-4cdf-9ec9-9c4dcbde097a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:44.119745 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:44.131513 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233d8f3-5f9c-4cdf-9ec9-9c4dcbde097a None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:20:44.255878 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca92a44-8784-4cda-9d95-d5e8626e1de4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:20:44.256538 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca92a44-8784-4cda-9d95-d5e8626e1de4 service nova] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:44.256786 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca92a44-8784-4cda-9d95-d5e8626e1de4 service nova] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:44.257060 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca92a44-8784-4cda-9d95-d5e8626e1de4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:44.258663 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-7ca92a44-8784-4cda-9d95-d5e8626e1de4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:44.364564 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca92a44-8784-4cda-9d95-d5e8626e1de4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:20:44.366187 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca92a44-8784-4cda-9d95-d5e8626e1de4 service nova] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:20:44.377865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:44.387687 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 691/1382] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:20:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:20:46.346392 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ecac68-d0ff-4836-9a5c-2b681c3843da None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:46.347340 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ecac68-d0ff-4836-9a5c-2b681c3843da None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:46.347340 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ecac68-d0ff-4836-9a5c-2b681c3843da None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:46.348596 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 692/1383] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:20:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:20:46.355206 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db9052-5809-4442-a389-91e78de93a5b None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:46.355693 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db9052-5809-4442-a389-91e78de93a5b None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:46.355804 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db9052-5809-4442-a389-91e78de93a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:46.424031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-d0db9052-5809-4442-a389-91e78de93a5b None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:20:46.425289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-d0db9052-5809-4442-a389-91e78de93a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:46.530535 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 692/1384] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:20:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:20:46.837866 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ecac68-d0ff-4836-9a5c-2b681c3843da None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:20:46.839676 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-83ecac68-d0ff-4836-9a5c-2b681c3843da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:46.934460 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:47.006861 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ecac68-d0ff-4836-9a5c-2b681c3843da None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:20:47.126318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40e57e-fbf8-4569-a787-226008a4ef56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:20:47.127127 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40e57e-fbf8-4569-a787-226008a4ef56 service nova] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:47.127503 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40e57e-fbf8-4569-a787-226008a4ef56 service nova] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:47.127882 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40e57e-fbf8-4569-a787-226008a4ef56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:47.130193 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-5e40e57e-fbf8-4569-a787-226008a4ef56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:47.189591 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:47.234847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e40e57e-fbf8-4569-a787-226008a4ef56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:20:47.236808 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e40e57e-fbf8-4569-a787-226008a4ef56 service nova] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:20:47.255322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 693/1385] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:20:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:20:49.245534 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded6ecb-094a-4dc6-8c32-a07056b8d236 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:49.246152 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded6ecb-094a-4dc6-8c32-a07056b8d236 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:49.246152 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded6ecb-094a-4dc6-8c32-a07056b8d236 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:49.247490 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 693/1386] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:20:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:20:49.253733 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5b39f-48ca-4e5f-ae07-840fae2e8ba9 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:49.254334 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5b39f-48ca-4e5f-ae07-840fae2e8ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:49.254334 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5b39f-48ca-4e5f-ae07-840fae2e8ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:49.335880 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-18f5b39f-48ca-4e5f-ae07-840fae2e8ba9 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:20:49.337067 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-18f5b39f-48ca-4e5f-ae07-840fae2e8ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:49.447984 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 694/1387] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:20:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:20:49.748348 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded6ecb-094a-4dc6-8c32-a07056b8d236 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:20:49.750145 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-cded6ecb-094a-4dc6-8c32-a07056b8d236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:49.835213 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:49.911319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded6ecb-094a-4dc6-8c32-a07056b8d236 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:20:50.035190 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e9995-a353-4348-b71e-d3c4d5ef102f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:20:50.035743 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e9995-a353-4348-b71e-d3c4d5ef102f service nova] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:50.036045 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e9995-a353-4348-b71e-d3c4d5ef102f service nova] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:50.036308 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e9995-a353-4348-b71e-d3c4d5ef102f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:50.037943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a57e9995-a353-4348-b71e-d3c4d5ef102f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:50.078558 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:50.132936 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57e9995-a353-4348-b71e-d3c4d5ef102f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:20:50.135332 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57e9995-a353-4348-b71e-d3c4d5ef102f service nova] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:20:50.167813 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 694/1388] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:20:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:20:52.227399 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e36250-a2bf-466b-bcba-8c67f6bb63b2 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:52.228036 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e36250-a2bf-466b-bcba-8c67f6bb63b2 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:52.228036 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e36250-a2bf-466b-bcba-8c67f6bb63b2 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:52.229474 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 695/1389] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:20:52.235149 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4842c26-5c53-454c-bbb3-0e205fb76c67 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:52.235648 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4842c26-5c53-454c-bbb3-0e205fb76c67 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:52.235648 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4842c26-5c53-454c-bbb3-0e205fb76c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:52.304392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-a4842c26-5c53-454c-bbb3-0e205fb76c67 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:20:52.305588 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a4842c26-5c53-454c-bbb3-0e205fb76c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:52.406777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 695/1390] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:20:52.654940 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf5c587-87b1-4f29-aa09-585d3d18148b None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:52.655219 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf5c587-87b1-4f29-aa09-585d3d18148b None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:52.655469 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf5c587-87b1-4f29-aa09-585d3d18148b None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:52.657024 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 696/1391] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:20:52.662320 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af296b05-74d8-4848-bc53-3b2968a6d448 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:52.662606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af296b05-74d8-4848-bc53-3b2968a6d448 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:52.662879 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af296b05-74d8-4848-bc53-3b2968a6d448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:52.679486 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:52.694267 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf5c587-87b1-4f29-aa09-585d3d18148b None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:20:52.696147 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-7bf5c587-87b1-4f29-aa09-585d3d18148b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:52.729546 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:52.737635 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-af296b05-74d8-4848-bc53-3b2968a6d448 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:20:52.738883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-af296b05-74d8-4848-bc53-3b2968a6d448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:52.836965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 696/1392] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:20:52.849695 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf5c587-87b1-4f29-aa09-585d3d18148b None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:20:52.951613 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:52.963866 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc2653-393b-4cc5-9321-56c74b3b0dac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:20:52.964512 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc2653-393b-4cc5-9321-56c74b3b0dac service nova] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:52.964793 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc2653-393b-4cc5-9321-56c74b3b0dac service nova] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:52.965062 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc2653-393b-4cc5-9321-56c74b3b0dac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:52.967055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-c9bc2653-393b-4cc5-9321-56c74b3b0dac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:53.060744 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bc2653-393b-4cc5-9321-56c74b3b0dac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:20:53.062371 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bc2653-393b-4cc5-9321-56c74b3b0dac service nova] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:20:53.080841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 697/1393] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:20:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:20:53.142557 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d316a5c-0e8b-48a9-af96-8738a4d97f8a None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:53.142842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d316a5c-0e8b-48a9-af96-8738a4d97f8a None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:53.143112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d316a5c-0e8b-48a9-af96-8738a4d97f8a None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:53.144870 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 697/1394] 10.222.0.18 () {54 vars in 967 bytes} [Tue Aug 23 04:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:20:53.149737 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81992496-de26-41f2-a320-1444d145ea9b None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:53.150076 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81992496-de26-41f2-a320-1444d145ea9b None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:53.150420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81992496-de26-41f2-a320-1444d145ea9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:53.241575 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-81992496-de26-41f2-a320-1444d145ea9b None None] MFA Rules not processed for user `4f1bf93cff6c46e0801bb827203cb5ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:20:53.242756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-81992496-de26-41f2-a320-1444d145ea9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:53.302759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:53.328010 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 698/1395] 10.222.0.18 () {58 vars in 1064 bytes} [Tue Aug 23 04:20:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:20:53.335929 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d316a5c-0e8b-48a9-af96-8738a4d97f8a None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:20:53.337442 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-8d316a5c-0e8b-48a9-af96-8738a4d97f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:53.438998 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:53.446544 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d316a5c-0e8b-48a9-af96-8738a4d97f8a None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:20:53.523817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2bb486-f7ad-4e15-a60b-bbca8fedf600 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f1bf93cff6c46e0801bb827203cb5ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:20:53.524362 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2bb486-f7ad-4e15-a60b-bbca8fedf600 service glance] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:53.524633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2bb486-f7ad-4e15-a60b-bbca8fedf600 service glance] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:53.524921 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2bb486-f7ad-4e15-a60b-bbca8fedf600 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:53.526502 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-1e2bb486-f7ad-4e15-a60b-bbca8fedf600 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:53.645764 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2bb486-f7ad-4e15-a60b-bbca8fedf600 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:20:53.647370 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2bb486-f7ad-4e15-a60b-bbca8fedf600 service glance] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:20:53.666647 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 698/1396] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:20:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:20:54.421587 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de97bd-ee48-49c1-9b64-94e956f19cf2 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:54.422128 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de97bd-ee48-49c1-9b64-94e956f19cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:54.422128 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de97bd-ee48-49c1-9b64-94e956f19cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:54.497132 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-f8de97bd-ee48-49c1-9b64-94e956f19cf2 None None] MFA Rules not processed for user `d3538271ebfd41aabe9f3da85feb7bca`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:20:54.498358 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-f8de97bd-ee48-49c1-9b64-94e956f19cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:54.586526 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 699/1397] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Aug 23 04:20:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:20:54.633415 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25896d-d2ff-476c-913f-b61c32ffa2bc None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:54.633934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25896d-d2ff-476c-913f-b61c32ffa2bc None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:54.633934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25896d-d2ff-476c-913f-b61c32ffa2bc None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:54.635378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 699/1398] 10.222.0.18 () {54 vars in 943 bytes} [Tue Aug 23 04:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:20:54.641657 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c5b00-3cf7-4b49-8257-0a23fc904310 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:54.641949 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c5b00-3cf7-4b49-8257-0a23fc904310 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:54.642226 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c5b00-3cf7-4b49-8257-0a23fc904310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:54.723517 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-1e5c5b00-3cf7-4b49-8257-0a23fc904310 None None] MFA Rules not processed for user `3f1418fd211a448da1a50175dce13b32`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:20:54.724661 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-1e5c5b00-3cf7-4b49-8257-0a23fc904310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:54.738023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:54.834468 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 700/1399] 10.222.0.18 () {58 vars in 1040 bytes} [Tue Aug 23 04:20:54 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:20:54.844859 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25896d-d2ff-476c-913f-b61c32ffa2bc None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:20:54.846424 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-ff25896d-d2ff-476c-913f-b61c32ffa2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:54.871320 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:54.989067 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25896d-d2ff-476c-913f-b61c32ffa2bc None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:20:55.020487 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a36da-7751-45ce-9137-43605e482aa0 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:55.020774 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a36da-7751-45ce-9137-43605e482aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:55.021046 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a36da-7751-45ce-9137-43605e482aa0 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:55.022450 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 701/1400] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:20:55.028139 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287c134-94de-4589-8c1d-3035605b654c None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:55.028425 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287c134-94de-4589-8c1d-3035605b654c None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:55.028700 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287c134-94de-4589-8c1d-3035605b654c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:55.095538 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01a948-21a4-479e-b22a-f7835bf57872 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f1418fd211a448da1a50175dce13b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:20:55.096188 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01a948-21a4-479e-b22a-f7835bf57872 service swift] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:55.096461 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01a948-21a4-479e-b22a-f7835bf57872 service swift] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:55.096726 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01a948-21a4-479e-b22a-f7835bf57872 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:55.097247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-4287c134-94de-4589-8c1d-3035605b654c None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:20:55.098374 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-8c01a948-21a4-479e-b22a-f7835bf57872 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:55.098469 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-4287c134-94de-4589-8c1d-3035605b654c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:55.106285 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:55.180544 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:55.185828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01a948-21a4-479e-b22a-f7835bf57872 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:20:55.187429 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01a948-21a4-479e-b22a-f7835bf57872 service swift] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:20:55.195517 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 700/1401] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Aug 23 04:20:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 04:20:55.204663 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 702/1402] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:20:55.499499 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01a948-21a4-479e-b22a-f7835bf57872 service swift] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:20:55.500674 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-8c01a948-21a4-479e-b22a-f7835bf57872 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:55.602171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01a948-21a4-479e-b22a-f7835bf57872 service swift] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:20:55.669209 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:55.735232 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc90913-bf65-4c10-bdc9-6c1d0bfbf20c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:20:55.735983 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc90913-bf65-4c10-bdc9-6c1d0bfbf20c service nova] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:55.736350 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc90913-bf65-4c10-bdc9-6c1d0bfbf20c service nova] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:55.736690 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc90913-bf65-4c10-bdc9-6c1d0bfbf20c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:55.738406 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-fbc90913-bf65-4c10-bdc9-6c1d0bfbf20c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:55.871787 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc90913-bf65-4c10-bdc9-6c1d0bfbf20c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:20:55.873362 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc90913-bf65-4c10-bdc9-6c1d0bfbf20c service nova] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:20:55.893877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 701/1403] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:20:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:20:57.834324 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73b62d-a131-4148-ab80-b7a703ad0faa None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:57.834921 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73b62d-a131-4148-ab80-b7a703ad0faa None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:57.834921 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73b62d-a131-4148-ab80-b7a703ad0faa None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:57.840335 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 703/1404] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:20:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:20:57.842557 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0084c-d339-4e63-8188-b6bce4ab1c7c None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:57.842975 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0084c-d339-4e63-8188-b6bce4ab1c7c None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:57.843157 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0084c-d339-4e63-8188-b6bce4ab1c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:57.915374 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-acd0084c-d339-4e63-8188-b6bce4ab1c7c None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:20:57.916680 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-acd0084c-d339-4e63-8188-b6bce4ab1c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:57.931379 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:58.032255 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 702/1405] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:20:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:20:58.317748 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73b62d-a131-4148-ab80-b7a703ad0faa None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:20:58.319607 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-ef73b62d-a131-4148-ab80-b7a703ad0faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:58.483994 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73b62d-a131-4148-ab80-b7a703ad0faa None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:20:58.549603 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:20:58.668009 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0eb930-1f24-4b83-a3b9-e14bef6bc2e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:20:58.668009 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0eb930-1f24-4b83-a3b9-e14bef6bc2e3 service nova] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:20:58.668009 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0eb930-1f24-4b83-a3b9-e14bef6bc2e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:20:58.668009 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0eb930-1f24-4b83-a3b9-e14bef6bc2e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:20:58.668511 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-dc0eb930-1f24-4b83-a3b9-e14bef6bc2e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:20:58.844035 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0eb930-1f24-4b83-a3b9-e14bef6bc2e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:20:58.845674 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0eb930-1f24-4b83-a3b9-e14bef6bc2e3 service nova] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:20:58.867767 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 704/1406] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:20:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:21:00.827380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069c316-2a68-4ae2-831c-fdb3967fafc8 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:00.828161 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069c316-2a68-4ae2-831c-fdb3967fafc8 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:00.828161 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069c316-2a68-4ae2-831c-fdb3967fafc8 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:00.829408 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 703/1407] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:21:00.834576 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8ea27-49b0-4701-ab25-f04148b1b49f None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:00.835003 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8ea27-49b0-4701-ab25-f04148b1b49f None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:00.835106 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8ea27-49b0-4701-ab25-f04148b1b49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:00.908793 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-11f8ea27-49b0-4701-ab25-f04148b1b49f None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:21:00.910031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-11f8ea27-49b0-4701-ab25-f04148b1b49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:00.931686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:01.035964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 705/1408] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:21:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:21:01.320231 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069c316-2a68-4ae2-831c-fdb3967fafc8 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:21:01.321770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-5069c316-2a68-4ae2-831c-fdb3967fafc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:01.479613 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:01.486237 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069c316-2a68-4ae2-831c-fdb3967fafc8 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:21:01.612805 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c74c53-2fbd-4d2a-96a9-553f69264876 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:21:01.613402 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c74c53-2fbd-4d2a-96a9-553f69264876 service nova] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:01.613692 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c74c53-2fbd-4d2a-96a9-553f69264876 service nova] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:01.613959 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c74c53-2fbd-4d2a-96a9-553f69264876 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:01.615560 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-56c74c53-2fbd-4d2a-96a9-553f69264876 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:01.715396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c74c53-2fbd-4d2a-96a9-553f69264876 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:21:01.717563 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c74c53-2fbd-4d2a-96a9-553f69264876 service nova] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:21:01.741817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 704/1409] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:21:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:21:11.029641 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=35 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:11.739423 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:17.114519 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641f622-a477-4dab-99d4-d35d3a7a2dd9 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:17.115179 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641f622-a477-4dab-99d4-d35d3a7a2dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:17.115179 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641f622-a477-4dab-99d4-d35d3a7a2dd9 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:17.116438 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 706/1410] 10.222.0.18 () {54 vars in 918 bytes} [Tue Aug 23 04:21:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:21:17.121896 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717dc86c-94e7-4bfe-94a5-94f67a1cbb49 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:17.122321 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717dc86c-94e7-4bfe-94a5-94f67a1cbb49 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:17.122420 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717dc86c-94e7-4bfe-94a5-94f67a1cbb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:17.201256 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-717dc86c-94e7-4bfe-94a5-94f67a1cbb49 None None] MFA Rules not processed for user `dd503fe3cf6d428e962c14ab864ee608`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:21:17.202440 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-717dc86c-94e7-4bfe-94a5-94f67a1cbb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:17.314336 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 705/1411] 10.222.0.18 () {58 vars in 1015 bytes} [Tue Aug 23 04:21:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:21:17.329149 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cea93-3357-4db0-91c6-07bdacfab9d4 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:17.329434 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cea93-3357-4db0-91c6-07bdacfab9d4 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:17.329707 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cea93-3357-4db0-91c6-07bdacfab9d4 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:17.331033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 707/1412] 10.222.0.18 () {54 vars in 918 bytes} [Tue Aug 23 04:21:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:21:17.336082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c0210-521d-407e-bac5-a5290a616089 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:17.336373 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c0210-521d-407e-bac5-a5290a616089 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:17.336643 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c0210-521d-407e-bac5-a5290a616089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:17.426983 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:17.430571 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-890c0210-521d-407e-bac5-a5290a616089 None None] MFA Rules not processed for user `cd7b11f479a54adc8d9b8641859b07a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:21:17.432748 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-890c0210-521d-407e-bac5-a5290a616089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:17.581971 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 706/1413] 10.222.0.18 () {58 vars in 1015 bytes} [Tue Aug 23 04:21:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:21:19.994816 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44934165-4c37-483d-811c-134155997206 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:19.995448 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44934165-4c37-483d-811c-134155997206 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:19.995448 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44934165-4c37-483d-811c-134155997206 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:19.996801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 04:21:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:21:20.004900 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aac075-ae8c-4b7a-a146-463fb2f1e632 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:20.005344 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aac075-ae8c-4b7a-a146-463fb2f1e632 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:20.005434 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aac075-ae8c-4b7a-a146-463fb2f1e632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:20.075596 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-42aac075-ae8c-4b7a-a146-463fb2f1e632 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:21:20.076832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-42aac075-ae8c-4b7a-a146-463fb2f1e632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:20.134420 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:20.183538 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 04:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:21:20.472551 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44934165-4c37-483d-811c-134155997206 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:21:20.475049 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-44934165-4c37-483d-811c-134155997206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:20.752342 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44934165-4c37-483d-811c-134155997206 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:21:20.826501 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:20.868532 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bdf49f-3b19-4c1b-9e61-b3e3909c9966 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:21:20.869197 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bdf49f-3b19-4c1b-9e61-b3e3909c9966 service nova] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:20.869467 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bdf49f-3b19-4c1b-9e61-b3e3909c9966 service nova] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:20.869747 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bdf49f-3b19-4c1b-9e61-b3e3909c9966 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:20.871365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-79bdf49f-3b19-4c1b-9e61-b3e3909c9966 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:20.960140 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bdf49f-3b19-4c1b-9e61-b3e3909c9966 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:21:20.961751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bdf49f-3b19-4c1b-9e61-b3e3909c9966 service nova] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:21:20.980784 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 709/1416] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:21:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:21:21.905049 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5e3f2-b453-4974-a939-6a2fb7c1cd71 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:21.905935 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5e3f2-b453-4974-a939-6a2fb7c1cd71 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:21.905935 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5e3f2-b453-4974-a939-6a2fb7c1cd71 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:21.907794 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 04:21:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:21:21.916173 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eadb9c-ef76-4012-9b70-8aae680c2c5b None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:21.916669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eadb9c-ef76-4012-9b70-8aae680c2c5b None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:21.916830 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eadb9c-ef76-4012-9b70-8aae680c2c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:22.004920 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-e3eadb9c-ef76-4012-9b70-8aae680c2c5b None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:21:22.006735 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-e3eadb9c-ef76-4012-9b70-8aae680c2c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:22.045807 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:22.121182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 04:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:21:22.278852 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c5e3f2-b453-4974-a939-6a2fb7c1cd71 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:21:22.280392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a8c5e3f2-b453-4974-a939-6a2fb7c1cd71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:22.456358 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c5e3f2-b453-4974-a939-6a2fb7c1cd71 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:21:22.500057 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:22.595684 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07613cc5-588b-48d8-9791-55687ec01381 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:21:22.596246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07613cc5-588b-48d8-9791-55687ec01381 service nova] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:22.596521 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07613cc5-588b-48d8-9791-55687ec01381 service nova] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:22.596791 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07613cc5-588b-48d8-9791-55687ec01381 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:22.598401 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-07613cc5-588b-48d8-9791-55687ec01381 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:22.690380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07613cc5-588b-48d8-9791-55687ec01381 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:21:22.691957 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07613cc5-588b-48d8-9791-55687ec01381 service nova] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:21:22.711282 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 709/1419] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:21:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:21:23.604228 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba279197-af3e-43df-ba56-9254900345d5 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:23.604835 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba279197-af3e-43df-ba56-9254900345d5 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:23.604835 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba279197-af3e-43df-ba56-9254900345d5 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:23.606159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 04:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:21:23.613263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b7391-5fc3-4748-b1d2-2ea246e747cb None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:23.613997 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b7391-5fc3-4748-b1d2-2ea246e747cb None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:23.614352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b7391-5fc3-4748-b1d2-2ea246e747cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:23.685492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:23.711529 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-778b7391-5fc3-4748-b1d2-2ea246e747cb None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:21:23.712785 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-778b7391-5fc3-4748-b1d2-2ea246e747cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:23.817477 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 04:21:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:21:23.971006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba279197-af3e-43df-ba56-9254900345d5 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:21:23.972482 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-ba279197-af3e-43df-ba56-9254900345d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:24.145674 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba279197-af3e-43df-ba56-9254900345d5 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:21:24.158096 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:24.269352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1252b-f5d9-4269-bc8b-d66776b8c5a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:21:24.269748 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1252b-f5d9-4269-bc8b-d66776b8c5a0 service nova] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:24.270016 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1252b-f5d9-4269-bc8b-d66776b8c5a0 service nova] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:24.270281 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1252b-f5d9-4269-bc8b-d66776b8c5a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:24.271907 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-bca1252b-f5d9-4269-bc8b-d66776b8c5a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:24.368519 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca1252b-f5d9-4269-bc8b-d66776b8c5a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:21:24.370166 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca1252b-f5d9-4269-bc8b-d66776b8c5a0 service nova] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:21:24.388407 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 712/1422] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:21:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:21:25.266916 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ffb88-0762-45b5-a9ae-f02596bc35d8 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:25.267550 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ffb88-0762-45b5-a9ae-f02596bc35d8 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:25.267550 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ffb88-0762-45b5-a9ae-f02596bc35d8 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:25.269595 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 04:21:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:21:25.278289 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27cb7f-4b2f-4cf8-ac93-3a3e3ecdd15b None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:25.278960 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27cb7f-4b2f-4cf8-ac93-3a3e3ecdd15b None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:25.279067 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27cb7f-4b2f-4cf8-ac93-3a3e3ecdd15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:25.309492 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:25.380105 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-ab27cb7f-4b2f-4cf8-ac93-3a3e3ecdd15b None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:21:25.381389 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-ab27cb7f-4b2f-4cf8-ac93-3a3e3ecdd15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:25.494835 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 04:21:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:21:25.643788 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ffb88-0762-45b5-a9ae-f02596bc35d8 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:21:25.645229 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-610ffb88-0762-45b5-a9ae-f02596bc35d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:25.778499 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:25.816424 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ffb88-0762-45b5-a9ae-f02596bc35d8 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:21:25.941678 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494c5fa-d355-4f47-bb3f-8d5b3733e556 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:21:25.942292 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494c5fa-d355-4f47-bb3f-8d5b3733e556 service nova] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:25.942571 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494c5fa-d355-4f47-bb3f-8d5b3733e556 service nova] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:25.942837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494c5fa-d355-4f47-bb3f-8d5b3733e556 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:25.944464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-b494c5fa-d355-4f47-bb3f-8d5b3733e556 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:26.025353 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:26.040643 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494c5fa-d355-4f47-bb3f-8d5b3733e556 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:21:26.042309 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494c5fa-d355-4f47-bb3f-8d5b3733e556 service nova] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:21:26.061539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 712/1425] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:21:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:21:26.918395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0a925-4308-4aa4-8b6c-476d68659b03 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:26.919249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0a925-4308-4aa4-8b6c-476d68659b03 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:26.919249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0a925-4308-4aa4-8b6c-476d68659b03 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:26.921451 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 04:21:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:21:26.929912 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286eb005-d00f-4f40-8b63-d9cae3b96877 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:26.930341 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286eb005-d00f-4f40-8b63-d9cae3b96877 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:26.930499 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286eb005-d00f-4f40-8b63-d9cae3b96877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:27.015093 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-286eb005-d00f-4f40-8b63-d9cae3b96877 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:21:27.016354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-286eb005-d00f-4f40-8b63-d9cae3b96877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:27.122215 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 04:21:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:21:27.274242 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd0a925-4308-4aa4-8b6c-476d68659b03 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:21:27.275719 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-7fd0a925-4308-4aa4-8b6c-476d68659b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:27.366684 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:27.440960 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd0a925-4308-4aa4-8b6c-476d68659b03 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:21:27.557267 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c03e1ad-84c6-4dc3-838d-d3f2c255e511 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:21:27.557622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c03e1ad-84c6-4dc3-838d-d3f2c255e511 service nova] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:27.557923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c03e1ad-84c6-4dc3-838d-d3f2c255e511 service nova] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:27.558185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c03e1ad-84c6-4dc3-838d-d3f2c255e511 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:27.559757 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-1c03e1ad-84c6-4dc3-838d-d3f2c255e511 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:27.622796 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:27.667051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c03e1ad-84c6-4dc3-838d-d3f2c255e511 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:21:27.668695 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c03e1ad-84c6-4dc3-838d-d3f2c255e511 service nova] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:21:27.687245 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 715/1428] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:21:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:21:28.497827 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2113034-ee05-40b7-959c-6175dabb4ffa None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:28.498321 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2113034-ee05-40b7-959c-6175dabb4ffa None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:28.498321 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2113034-ee05-40b7-959c-6175dabb4ffa None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:28.499754 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 04:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:21:28.505184 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4b228-e5eb-47f3-aefd-ac20c434977b None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:28.505631 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4b228-e5eb-47f3-aefd-ac20c434977b None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:28.505727 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4b228-e5eb-47f3-aefd-ac20c434977b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:28.589644 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-9fb4b228-e5eb-47f3-aefd-ac20c434977b None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:21:28.590860 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-9fb4b228-e5eb-47f3-aefd-ac20c434977b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:28.700558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 04:21:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:21:28.855527 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2113034-ee05-40b7-959c-6175dabb4ffa None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:21:28.858007 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-e2113034-ee05-40b7-959c-6175dabb4ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:28.922160 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:29.033134 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2113034-ee05-40b7-959c-6175dabb4ffa None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:21:29.159156 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d79633-ed2f-4069-8188-49a2233e68ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:21:29.159771 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d79633-ed2f-4069-8188-49a2233e68ff service nova] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:29.160089 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d79633-ed2f-4069-8188-49a2233e68ff service nova] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:29.160360 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d79633-ed2f-4069-8188-49a2233e68ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:29.161974 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a7d79633-ed2f-4069-8188-49a2233e68ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:29.169124 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:29.261212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d79633-ed2f-4069-8188-49a2233e68ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:21:29.262847 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d79633-ed2f-4069-8188-49a2233e68ff service nova] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:21:29.282632 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 715/1431] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:21:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:21:30.110328 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f046b-7dc4-4ba7-9605-13043fe61f47 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:30.111364 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f046b-7dc4-4ba7-9605-13043fe61f47 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:30.111364 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f046b-7dc4-4ba7-9605-13043fe61f47 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:30.113499 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 04:21:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:21:30.121820 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36aa5b-d8b3-4a9f-9669-24e10a36a272 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:30.122278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36aa5b-d8b3-4a9f-9669-24e10a36a272 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:30.122443 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36aa5b-d8b3-4a9f-9669-24e10a36a272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:30.210201 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-2f36aa5b-d8b3-4a9f-9669-24e10a36a272 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:21:30.211429 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-2f36aa5b-d8b3-4a9f-9669-24e10a36a272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:30.287712 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:30.328976 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 04:21:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:21:30.481812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f046b-7dc4-4ba7-9605-13043fe61f47 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:21:30.483367 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-612f046b-7dc4-4ba7-9605-13043fe61f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:30.502226 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:30.640104 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f046b-7dc4-4ba7-9605-13043fe61f47 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:21:30.736207 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:30.766146 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71e07a-9881-4e94-af48-ae9ca885bdfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:21:30.766779 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71e07a-9881-4e94-af48-ae9ca885bdfa service nova] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:30.767059 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71e07a-9881-4e94-af48-ae9ca885bdfa service nova] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:30.767331 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71e07a-9881-4e94-af48-ae9ca885bdfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:30.768977 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-ca71e07a-9881-4e94-af48-ae9ca885bdfa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:30.875272 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71e07a-9881-4e94-af48-ae9ca885bdfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:21:30.877264 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71e07a-9881-4e94-af48-ae9ca885bdfa service nova] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:21:30.896338 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 718/1434] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:21:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:21:31.707266 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc18efb-7950-4af5-9552-b673b8cc3f99 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:31.708147 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc18efb-7950-4af5-9552-b673b8cc3f99 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:31.708147 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc18efb-7950-4af5-9552-b673b8cc3f99 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:31.710284 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 04:21:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:21:31.719148 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc831b2b-ae49-44ef-b24c-3b9f788cec7e None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:31.719619 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc831b2b-ae49-44ef-b24c-3b9f788cec7e None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:31.719737 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc831b2b-ae49-44ef-b24c-3b9f788cec7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:31.805593 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-cc831b2b-ae49-44ef-b24c-3b9f788cec7e None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:21:31.806800 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-cc831b2b-ae49-44ef-b24c-3b9f788cec7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:31.848878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:31.913521 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 04:21:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:21:32.067681 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc18efb-7950-4af5-9552-b673b8cc3f99 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:21:32.069148 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-7cc18efb-7950-4af5-9552-b673b8cc3f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:32.253221 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc18efb-7950-4af5-9552-b673b8cc3f99 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:21:32.312932 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:32.375397 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d172f-d63a-4c77-9829-6ff8c8f8a88e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:21:32.375949 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d172f-d63a-4c77-9829-6ff8c8f8a88e service nova] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:32.376254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d172f-d63a-4c77-9829-6ff8c8f8a88e service nova] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:32.376515 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d172f-d63a-4c77-9829-6ff8c8f8a88e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:32.378116 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-b38d172f-d63a-4c77-9829-6ff8c8f8a88e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:32.476936 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38d172f-d63a-4c77-9829-6ff8c8f8a88e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:21:32.478592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38d172f-d63a-4c77-9829-6ff8c8f8a88e service nova] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:21:32.496675 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 718/1437] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:21:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:21:33.313705 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf3215-fd2a-4866-98dd-0e1897b04290 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:33.314591 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf3215-fd2a-4866-98dd-0e1897b04290 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:33.314591 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf3215-fd2a-4866-98dd-0e1897b04290 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:33.316961 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 04:21:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:21:33.323073 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e277a-a260-47cd-bbbc-5bb117c76052 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:33.323511 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e277a-a260-47cd-bbbc-5bb117c76052 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:33.323618 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e277a-a260-47cd-bbbc-5bb117c76052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:33.405698 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-e59e277a-a260-47cd-bbbc-5bb117c76052 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:21:33.406918 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-e59e277a-a260-47cd-bbbc-5bb117c76052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:33.409138 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:33.516394 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 04:21:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:21:33.670258 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf3215-fd2a-4866-98dd-0e1897b04290 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:21:33.671840 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-0dcf3215-fd2a-4866-98dd-0e1897b04290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:33.830599 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf3215-fd2a-4866-98dd-0e1897b04290 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:21:33.863602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:33.960164 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5828751-0001-490f-af6e-45be3cb56b56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:21:33.960678 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5828751-0001-490f-af6e-45be3cb56b56 service nova] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:33.960955 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5828751-0001-490f-af6e-45be3cb56b56 service nova] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:33.961223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5828751-0001-490f-af6e-45be3cb56b56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:33.962831 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-d5828751-0001-490f-af6e-45be3cb56b56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:34.061760 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5828751-0001-490f-af6e-45be3cb56b56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:21:34.063359 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5828751-0001-490f-af6e-45be3cb56b56 service nova] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:21:34.081687 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 721/1440] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:21:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:21:35.900169 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548d3f0-ae3b-4b35-91da-5b6a0f3a15f4 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:35.900937 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548d3f0-ae3b-4b35-91da-5b6a0f3a15f4 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:35.901532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548d3f0-ae3b-4b35-91da-5b6a0f3a15f4 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:35.903256 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 04:21:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:21:35.909035 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed216c4e-0644-49a5-8f65-2139915bcd82 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:35.909501 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed216c4e-0644-49a5-8f65-2139915bcd82 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:35.909611 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed216c4e-0644-49a5-8f65-2139915bcd82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:35.955949 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:35.995145 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-ed216c4e-0644-49a5-8f65-2139915bcd82 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:21:35.996454 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-ed216c4e-0644-49a5-8f65-2139915bcd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:36.109532 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 04:21:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:21:36.401651 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548d3f0-ae3b-4b35-91da-5b6a0f3a15f4 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:21:36.403197 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a548d3f0-ae3b-4b35-91da-5b6a0f3a15f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:36.525420 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548d3f0-ae3b-4b35-91da-5b6a0f3a15f4 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:21:36.560503 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:36.603686 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e493b-a65c-4727-a7c2-33e233fceaef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:21:36.604340 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e493b-a65c-4727-a7c2-33e233fceaef service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:36.604716 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e493b-a65c-4727-a7c2-33e233fceaef service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:36.604989 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e493b-a65c-4727-a7c2-33e233fceaef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:36.606596 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-d63e493b-a65c-4727-a7c2-33e233fceaef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:36.732111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63e493b-a65c-4727-a7c2-33e233fceaef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:21:36.733578 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63e493b-a65c-4727-a7c2-33e233fceaef service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:21:36.752745 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 721/1443] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:21:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:21:38.800191 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde0e08-b0dc-4803-8a0b-4cc08fb0a9ff None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:38.800805 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde0e08-b0dc-4803-8a0b-4cc08fb0a9ff None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:38.800805 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde0e08-b0dc-4803-8a0b-4cc08fb0a9ff None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:38.802194 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 04:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:21:38.808505 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2a78c-f853-494b-a5a3-9d881110ecb0 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:38.808928 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2a78c-f853-494b-a5a3-9d881110ecb0 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:38.809031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2a78c-f853-494b-a5a3-9d881110ecb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:38.884886 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-c9e2a78c-f853-494b-a5a3-9d881110ecb0 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:21:38.886174 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-c9e2a78c-f853-494b-a5a3-9d881110ecb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:38.902586 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:38.999000 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 04:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:21:39.283728 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde0e08-b0dc-4803-8a0b-4cc08fb0a9ff None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:21:39.286248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-edde0e08-b0dc-4803-8a0b-4cc08fb0a9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:39.411135 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde0e08-b0dc-4803-8a0b-4cc08fb0a9ff None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:21:39.425087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:39.517149 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55dc39-8acd-4a61-aebf-a39ff0cdc9e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:21:39.517800 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55dc39-8acd-4a61-aebf-a39ff0cdc9e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:39.518067 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55dc39-8acd-4a61-aebf-a39ff0cdc9e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:39.518339 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55dc39-8acd-4a61-aebf-a39ff0cdc9e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:39.519954 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-fb55dc39-8acd-4a61-aebf-a39ff0cdc9e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:39.641840 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55dc39-8acd-4a61-aebf-a39ff0cdc9e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:21:39.643757 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55dc39-8acd-4a61-aebf-a39ff0cdc9e4 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:21:39.669618 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 724/1446] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:21:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:21:42.146103 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18a68b-28a8-42d3-8679-407e509c2b41 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:42.146677 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18a68b-28a8-42d3-8679-407e509c2b41 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:42.146677 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18a68b-28a8-42d3-8679-407e509c2b41 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:42.148044 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 04:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:21:42.154442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8d848-c656-42f9-ad0d-2c065065cc6c None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:42.154896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8d848-c656-42f9-ad0d-2c065065cc6c None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:42.154997 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8d848-c656-42f9-ad0d-2c065065cc6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:42.205374 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:42.227255 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-bff8d848-c656-42f9-ad0d-2c065065cc6c None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:21:42.228498 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-bff8d848-c656-42f9-ad0d-2c065065cc6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:42.351878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 04:21:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:21:42.639665 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18a68b-28a8-42d3-8679-407e509c2b41 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:21:42.641263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-9e18a68b-28a8-42d3-8679-407e509c2b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:42.788398 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18a68b-28a8-42d3-8679-407e509c2b41 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:21:42.830772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:42.875313 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71baba18-ca13-42a1-8bd1-20831a783e5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:21:42.876440 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71baba18-ca13-42a1-8bd1-20831a783e5a service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:42.876917 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71baba18-ca13-42a1-8bd1-20831a783e5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:42.877442 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71baba18-ca13-42a1-8bd1-20831a783e5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:42.880498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-71baba18-ca13-42a1-8bd1-20831a783e5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:43.017290 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71baba18-ca13-42a1-8bd1-20831a783e5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:21:43.018719 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71baba18-ca13-42a1-8bd1-20831a783e5a service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:21:43.038528 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 724/1449] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:21:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:21:45.224984 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1ee07-3ca2-4bea-a4dd-d8c519621a77 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:45.225626 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1ee07-3ca2-4bea-a4dd-d8c519621a77 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:45.225626 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1ee07-3ca2-4bea-a4dd-d8c519621a77 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:45.226982 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 04:21:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:21:45.232751 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd8e16-547b-41dc-9be5-adf38444b58a None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:45.233255 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd8e16-547b-41dc-9be5-adf38444b58a None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:45.233255 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd8e16-547b-41dc-9be5-adf38444b58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:45.314390 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-1bcd8e16-547b-41dc-9be5-adf38444b58a None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:21:45.315597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-1bcd8e16-547b-41dc-9be5-adf38444b58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:45.345207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:45.445646 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 04:21:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:21:45.760796 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1ee07-3ca2-4bea-a4dd-d8c519621a77 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:21:45.763271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-55e1ee07-3ca2-4bea-a4dd-d8c519621a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:45.877807 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1ee07-3ca2-4bea-a4dd-d8c519621a77 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:21:45.900032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:45.962669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f1c8a-b4d2-43e0-9584-a9f3331f2bb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:21:45.963508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f1c8a-b4d2-43e0-9584-a9f3331f2bb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:45.964194 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f1c8a-b4d2-43e0-9584-a9f3331f2bb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:45.964648 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f1c8a-b4d2-43e0-9584-a9f3331f2bb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:45.967075 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-d32f1c8a-b4d2-43e0-9584-a9f3331f2bb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:46.091388 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f1c8a-b4d2-43e0-9584-a9f3331f2bb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:21:46.093137 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f1c8a-b4d2-43e0-9584-a9f3331f2bb7 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:21:46.112738 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 727/1452] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:21:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:21:48.056815 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2233a-08af-4b37-87e4-92884852d6a0 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:48.057661 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2233a-08af-4b37-87e4-92884852d6a0 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:48.057661 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2233a-08af-4b37-87e4-92884852d6a0 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:48.058773 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 04:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:21:48.065270 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c465e-a86a-4eab-bba5-9881829dd919 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:48.066028 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c465e-a86a-4eab-bba5-9881829dd919 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:48.066219 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c465e-a86a-4eab-bba5-9881829dd919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:48.143864 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:48.147112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-093c465e-a86a-4eab-bba5-9881829dd919 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:21:48.149619 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-093c465e-a86a-4eab-bba5-9881829dd919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:48.274851 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 04:21:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:21:48.559680 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2233a-08af-4b37-87e4-92884852d6a0 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:21:48.562047 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-2ab2233a-08af-4b37-87e4-92884852d6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:48.689933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2233a-08af-4b37-87e4-92884852d6a0 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:21:48.741852 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:48.768563 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6218994-288e-44c9-9ca0-4a223469be7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:21:48.769175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6218994-288e-44c9-9ca0-4a223469be7d service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:48.769456 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6218994-288e-44c9-9ca0-4a223469be7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:48.769725 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6218994-288e-44c9-9ca0-4a223469be7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:48.771332 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-d6218994-288e-44c9-9ca0-4a223469be7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:48.898637 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6218994-288e-44c9-9ca0-4a223469be7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:21:48.900176 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6218994-288e-44c9-9ca0-4a223469be7d service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:21:48.918944 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 727/1455] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:21:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:21:50.877369 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d342d9-5bd5-419d-b399-17952135a73e None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:50.877929 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d342d9-5bd5-419d-b399-17952135a73e None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:50.877929 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d342d9-5bd5-419d-b399-17952135a73e None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:50.879184 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 04:21:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:21:50.885959 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203c75e1-149b-416f-a09b-ee2be9da4b23 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:50.886652 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203c75e1-149b-416f-a09b-ee2be9da4b23 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:50.886826 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203c75e1-149b-416f-a09b-ee2be9da4b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:50.968501 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-203c75e1-149b-416f-a09b-ee2be9da4b23 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:21:50.969726 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-203c75e1-149b-416f-a09b-ee2be9da4b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:51.002822 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:51.078622 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 04:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:21:51.369290 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d342d9-5bd5-419d-b399-17952135a73e None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:21:51.371129 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-a1d342d9-5bd5-419d-b399-17952135a73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:51.491874 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d342d9-5bd5-419d-b399-17952135a73e None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:21:51.521194 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:51.569467 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b00f9f-8c81-43b3-a3a0-6cdfee0dda06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:21:51.570085 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b00f9f-8c81-43b3-a3a0-6cdfee0dda06 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:51.570359 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b00f9f-8c81-43b3-a3a0-6cdfee0dda06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:51.570635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b00f9f-8c81-43b3-a3a0-6cdfee0dda06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:51.572286 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-a9b00f9f-8c81-43b3-a3a0-6cdfee0dda06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:51.689500 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b00f9f-8c81-43b3-a3a0-6cdfee0dda06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:21:51.691756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b00f9f-8c81-43b3-a3a0-6cdfee0dda06 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:21:51.710617 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 730/1458] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:21:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:21:53.714396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545cd6a-44ab-4dac-b5f6-dbcf817b14a8 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:53.715291 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545cd6a-44ab-4dac-b5f6-dbcf817b14a8 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:53.715291 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545cd6a-44ab-4dac-b5f6-dbcf817b14a8 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:53.717883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 04:21:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:21:53.725980 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72c876-ef60-433d-83fe-fd33fab92db6 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:53.726433 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72c876-ef60-433d-83fe-fd33fab92db6 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:53.726537 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72c876-ef60-433d-83fe-fd33fab92db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:53.795332 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-5a72c876-ef60-433d-83fe-fd33fab92db6 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:21:53.796610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-5a72c876-ef60-433d-83fe-fd33fab92db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:53.805128 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:53.912282 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 04:21:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:21:54.211268 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545cd6a-44ab-4dac-b5f6-dbcf817b14a8 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:21:54.213827 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-0545cd6a-44ab-4dac-b5f6-dbcf817b14a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:54.342304 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545cd6a-44ab-4dac-b5f6-dbcf817b14a8 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:21:54.404880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:54.423865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2b9b5-8fc7-4562-82a6-94844e1d2f7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:21:54.424505 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2b9b5-8fc7-4562-82a6-94844e1d2f7f service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:54.424778 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2b9b5-8fc7-4562-82a6-94844e1d2f7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:54.425046 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2b9b5-8fc7-4562-82a6-94844e1d2f7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:54.426639 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-7fa2b9b5-8fc7-4562-82a6-94844e1d2f7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:54.543779 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa2b9b5-8fc7-4562-82a6-94844e1d2f7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:21:54.545263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa2b9b5-8fc7-4562-82a6-94844e1d2f7f service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:21:54.563239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 730/1461] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:21:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:21:56.497643 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92417c-efd3-485d-b804-b3a080e7a1c8 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:56.498450 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92417c-efd3-485d-b804-b3a080e7a1c8 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:56.498450 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92417c-efd3-485d-b804-b3a080e7a1c8 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:56.500570 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 04:21:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:21:56.506550 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b93cb4-9a99-4c27-9db3-4462ef912ec8 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:56.507182 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b93cb4-9a99-4c27-9db3-4462ef912ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:56.507348 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b93cb4-9a99-4c27-9db3-4462ef912ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:56.577072 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-f0b93cb4-9a99-4c27-9db3-4462ef912ec8 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:21:56.578244 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-f0b93cb4-9a99-4c27-9db3-4462ef912ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:56.619033 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:56.685107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 04:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:21:56.974282 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92417c-efd3-485d-b804-b3a080e7a1c8 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:21:56.976809 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-2a92417c-efd3-485d-b804-b3a080e7a1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:57.104081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92417c-efd3-485d-b804-b3a080e7a1c8 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:21:57.144426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:57.181805 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dece9e-aa09-4f89-b035-35bf25a54501 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:21:57.182473 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dece9e-aa09-4f89-b035-35bf25a54501 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:57.182772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dece9e-aa09-4f89-b035-35bf25a54501 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:57.183041 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dece9e-aa09-4f89-b035-35bf25a54501 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:57.184702 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-86dece9e-aa09-4f89-b035-35bf25a54501 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:57.299829 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dece9e-aa09-4f89-b035-35bf25a54501 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:21:57.301262 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dece9e-aa09-4f89-b035-35bf25a54501 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:21:57.321881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 733/1464] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:21:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:21:59.212945 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877aab8-9442-4df4-bc28-68004226c67f None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:59.213843 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877aab8-9442-4df4-bc28-68004226c67f None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:59.213843 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877aab8-9442-4df4-bc28-68004226c67f None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:59.215409 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 04:21:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:21:59.222212 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8137e0-4034-4b1a-a5a9-951a82f28084 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:59.222666 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8137e0-4034-4b1a-a5a9-951a82f28084 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:59.222760 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8137e0-4034-4b1a-a5a9-951a82f28084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:59.318600 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-8e8137e0-4034-4b1a-a5a9-951a82f28084 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:21:59.319709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-8e8137e0-4034-4b1a-a5a9-951a82f28084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:59.337865 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:59.427244 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 04:21:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:21:59.708719 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877aab8-9442-4df4-bc28-68004226c67f None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:21:59.711263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-5877aab8-9442-4df4-bc28-68004226c67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:21:59.835657 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877aab8-9442-4df4-bc28-68004226c67f None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:21:59.908176 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:21:59.916914 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6375e86-ceaf-4784-8691-1b21bacea130 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:21:59.917539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6375e86-ceaf-4784-8691-1b21bacea130 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:21:59.917811 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6375e86-ceaf-4784-8691-1b21bacea130 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:21:59.918080 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6375e86-ceaf-4784-8691-1b21bacea130 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:21:59.919672 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-f6375e86-ceaf-4784-8691-1b21bacea130 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:22:00.037399 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6375e86-ceaf-4784-8691-1b21bacea130 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:22:00.038742 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6375e86-ceaf-4784-8691-1b21bacea130 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:22:00.057126 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 733/1467] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:21:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:22:01.975481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b738db-d99d-4ac6-93a4-22e3a493d99e None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:22:01.976518 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b738db-d99d-4ac6-93a4-22e3a493d99e None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:22:01.976518 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b738db-d99d-4ac6-93a4-22e3a493d99e None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:22:01.978530 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 04:22:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:22:01.984444 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6a7ac-154e-4abe-9859-0e0b945ae3e3 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:22:01.984952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6a7ac-154e-4abe-9859-0e0b945ae3e3 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:22:01.984952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6a7ac-154e-4abe-9859-0e0b945ae3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:22:02.052648 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-87b6a7ac-154e-4abe-9859-0e0b945ae3e3 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:22:02.053844 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-87b6a7ac-154e-4abe-9859-0e0b945ae3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:22:02.104787 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:22:02.159673 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 04:22:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:22:02.450691 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b738db-d99d-4ac6-93a4-22e3a493d99e None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:22:02.452315 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-c0b738db-d99d-4ac6-93a4-22e3a493d99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:22:02.568580 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b738db-d99d-4ac6-93a4-22e3a493d99e None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:22:02.617768 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:22:02.646378 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a299931-a80c-4353-b178-140e0425452b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:22:02.646986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a299931-a80c-4353-b178-140e0425452b service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:22:02.647270 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a299931-a80c-4353-b178-140e0425452b service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:22:02.647532 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a299931-a80c-4353-b178-140e0425452b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:22:02.649154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-2a299931-a80c-4353-b178-140e0425452b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:22:02.772216 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a299931-a80c-4353-b178-140e0425452b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:22:02.773615 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a299931-a80c-4353-b178-140e0425452b service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:22:02.795731 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 736/1470] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:22:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:22:12.157781 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=22 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:22:12.792870 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=66 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:22:22.374168 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3dd16d-ab67-400e-9a28-877815019739 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:22:22.374742 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3dd16d-ab67-400e-9a28-877815019739 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:22:22.375000 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3dd16d-ab67-400e-9a28-877815019739 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:22:22.377030 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 735/1471] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:22:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:22:22.385294 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633db2a-fb4a-437c-aee0-5543407dbb73 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:22:22.385908 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633db2a-fb4a-437c-aee0-5543407dbb73 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:22:22.385908 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633db2a-fb4a-437c-aee0-5543407dbb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:22:22.463766 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-e633db2a-fb4a-437c-aee0-5543407dbb73 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:22:22.465110 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-e633db2a-fb4a-437c-aee0-5543407dbb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:22:22.591905 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 737/1472] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:22:22.913395 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555371e-f690-4786-8e63-c9e81a01f79e None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:22:22.913898 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555371e-f690-4786-8e63-c9e81a01f79e None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:22:22.914345 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555371e-f690-4786-8e63-c9e81a01f79e None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:22:22.916278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 736/1473] 10.222.0.18 () {54 vars in 967 bytes} [Tue Aug 23 04:22:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:22:22.921741 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3039a-5456-4118-a88b-22d11ab5b589 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:22:22.922033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3039a-5456-4118-a88b-22d11ab5b589 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:22:22.922305 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3039a-5456-4118-a88b-22d11ab5b589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:22:23.020997 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-dba3039a-5456-4118-a88b-22d11ab5b589 None None] MFA Rules not processed for user `4f1bf93cff6c46e0801bb827203cb5ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:22:23.022265 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-dba3039a-5456-4118-a88b-22d11ab5b589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:22:23.056707 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:22:23.126280 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 738/1474] 10.222.0.18 () {58 vars in 1064 bytes} [Tue Aug 23 04:22:22 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:22:23.137514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555371e-f690-4786-8e63-c9e81a01f79e None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:22:23.138960 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-7555371e-f690-4786-8e63-c9e81a01f79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:22:23.256553 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555371e-f690-4786-8e63-c9e81a01f79e None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:22:23.351997 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ab1fb-22ab-4b96-aa02-e4e04841154b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f1bf93cff6c46e0801bb827203cb5ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:22:23.352855 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ab1fb-22ab-4b96-aa02-e4e04841154b service glance] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:22:23.353219 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ab1fb-22ab-4b96-aa02-e4e04841154b service glance] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:22:23.353572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ab1fb-22ab-4b96-aa02-e4e04841154b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:22:23.355316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-c95ab1fb-22ab-4b96-aa02-e4e04841154b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:22:23.414543 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:22:23.479256 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95ab1fb-22ab-4b96-aa02-e4e04841154b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:22:23.480741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95ab1fb-22ab-4b96-aa02-e4e04841154b service glance] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:22:23.501426 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 737/1475] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:22:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:22:25.489090 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23629831-3da2-4a70-943a-c667c878886c None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:22:25.489610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23629831-3da2-4a70-943a-c667c878886c None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:22:25.489719 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23629831-3da2-4a70-943a-c667c878886c None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:22:25.491021 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 739/1476] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:22:25.496729 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c76bf2-72e9-4b9a-8942-55488c48a410 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:22:25.497215 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c76bf2-72e9-4b9a-8942-55488c48a410 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:22:25.497215 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c76bf2-72e9-4b9a-8942-55488c48a410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:22:25.574655 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-45c76bf2-72e9-4b9a-8942-55488c48a410 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:22:25.575879 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-45c76bf2-72e9-4b9a-8942-55488c48a410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:22:25.682772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 738/1477] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:22:25.970558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23629831-3da2-4a70-943a-c667c878886c None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:22:25.972054 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-23629831-3da2-4a70-943a-c667c878886c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:22:26.116056 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23629831-3da2-4a70-943a-c667c878886c None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:22:26.188191 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:22:26.237616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ba2fe-8f25-473b-851f-f62616ff3e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:22:26.238147 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ba2fe-8f25-473b-851f-f62616ff3e9a admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:22:26.238420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ba2fe-8f25-473b-851f-f62616ff3e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:22:26.238682 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ba2fe-8f25-473b-851f-f62616ff3e9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:22:26.239333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ba2fe-8f25-473b-851f-f62616ff3e9a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:22:26.241463 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: WARNING py.warnings [None req-2c1ba2fe-8f25-473b-851f-f62616ff3e9a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 04:22:26.241463 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: warnings.warn(msg) Aug 23 04:22:26.241463 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: Aug 23 04:22:26.242574 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ba2fe-8f25-473b-851f-f62616ff3e9a admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:22:26.261931 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 740/1478] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Aug 23 04:22:25 2022] GET /identity/v3/projects => generated 2553 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:22:28.169456 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676b17e-0255-4854-94d6-6934c6146933 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:22:28.170034 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676b17e-0255-4854-94d6-6934c6146933 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:22:28.170034 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676b17e-0255-4854-94d6-6934c6146933 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:22:28.171376 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 739/1479] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:22:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:22:28.178045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e68306-069d-4c46-a4aa-d456753ce99e None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:22:28.178492 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e68306-069d-4c46-a4aa-d456753ce99e None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:22:28.178602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e68306-069d-4c46-a4aa-d456753ce99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:22:28.255417 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-e0e68306-069d-4c46-a4aa-d456753ce99e None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:22:28.256687 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-e0e68306-069d-4c46-a4aa-d456753ce99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:22:28.360757 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 741/1480] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:22:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:22:28.659975 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8676b17e-0255-4854-94d6-6934c6146933 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:22:28.662059 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-8676b17e-0255-4854-94d6-6934c6146933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:22:28.673784 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:22:28.824186 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8676b17e-0255-4854-94d6-6934c6146933 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:22:28.908456 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:22:28.950495 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83706578-8c14-4bbd-8ffb-2047e2b5bdb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:22:28.951110 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83706578-8c14-4bbd-8ffb-2047e2b5bdb1 service nova] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:22:28.951386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83706578-8c14-4bbd-8ffb-2047e2b5bdb1 service nova] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:22:28.951646 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83706578-8c14-4bbd-8ffb-2047e2b5bdb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:22:28.953452 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-83706578-8c14-4bbd-8ffb-2047e2b5bdb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:22:29.044350 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83706578-8c14-4bbd-8ffb-2047e2b5bdb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:22:29.045946 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83706578-8c14-4bbd-8ffb-2047e2b5bdb1 service nova] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:22:29.065461 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 740/1481] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:22:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:22:31.169698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95dece-ed4d-4e59-ae7a-3e87eed96d79 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:22:31.170405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95dece-ed4d-4e59-ae7a-3e87eed96d79 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:22:31.170405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95dece-ed4d-4e59-ae7a-3e87eed96d79 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:22:31.171597 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 742/1482] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:22:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:22:31.178163 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f285d-c081-4cb7-a8fc-e506d6b2cb7d None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:22:31.178872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f285d-c081-4cb7-a8fc-e506d6b2cb7d None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:22:31.179011 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f285d-c081-4cb7-a8fc-e506d6b2cb7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:22:31.264635 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-314f285d-c081-4cb7-a8fc-e506d6b2cb7d None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:22:31.266427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-314f285d-c081-4cb7-a8fc-e506d6b2cb7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:22:31.327318 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:22:31.390047 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 741/1483] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:22:31.686736 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95dece-ed4d-4e59-ae7a-3e87eed96d79 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:22:31.689234 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-5c95dece-ed4d-4e59-ae7a-3e87eed96d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:22:31.718212 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:22:31.808061 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95dece-ed4d-4e59-ae7a-3e87eed96d79 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:22:31.885583 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18feea-f4bc-495e-9b91-51d94ac8ecb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:22:31.886169 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18feea-f4bc-495e-9b91-51d94ac8ecb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:22:31.886448 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18feea-f4bc-495e-9b91-51d94ac8ecb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:22:31.886717 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18feea-f4bc-495e-9b91-51d94ac8ecb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:22:31.888353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-1a18feea-f4bc-495e-9b91-51d94ac8ecb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:22:31.965643 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:22:32.002300 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a18feea-f4bc-495e-9b91-51d94ac8ecb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:22:32.003816 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a18feea-f4bc-495e-9b91-51d94ac8ecb4 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:22:32.022903 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 743/1484] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:22:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:22:34.017320 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69918c-ca22-4d39-8a7d-11ef70487c5d None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:22:34.018148 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69918c-ca22-4d39-8a7d-11ef70487c5d None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:22:34.018148 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69918c-ca22-4d39-8a7d-11ef70487c5d None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:22:34.020316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 742/1485] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:22:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:22:34.029449 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad1efc-7b6d-4288-b834-5de948e45e50 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:22:34.030139 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad1efc-7b6d-4288-b834-5de948e45e50 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:22:34.030549 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad1efc-7b6d-4288-b834-5de948e45e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:22:34.101544 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-83ad1efc-7b6d-4288-b834-5de948e45e50 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:22:34.102775 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-83ad1efc-7b6d-4288-b834-5de948e45e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:22:34.214550 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 744/1486] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:22:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:22:34.505115 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69918c-ca22-4d39-8a7d-11ef70487c5d None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:22:34.507600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-5d69918c-ca22-4d39-8a7d-11ef70487c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:22:34.632593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69918c-ca22-4d39-8a7d-11ef70487c5d None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:22:34.716910 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:22:34.719794 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e373a-7447-43f9-ae98-f34a75103026 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:22:34.720753 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e373a-7447-43f9-ae98-f34a75103026 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:22:34.721175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e373a-7447-43f9-ae98-f34a75103026 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:22:34.721602 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e373a-7447-43f9-ae98-f34a75103026 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:22:34.724782 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-f64e373a-7447-43f9-ae98-f34a75103026 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:22:34.849990 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e373a-7447-43f9-ae98-f34a75103026 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:22:34.851419 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e373a-7447-43f9-ae98-f34a75103026 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:22:34.869743 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 743/1487] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:22:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:22:37.003518 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ee434-f43a-4e7e-8922-8bee0d265677 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:22:37.004154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ee434-f43a-4e7e-8922-8bee0d265677 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:22:37.004154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ee434-f43a-4e7e-8922-8bee0d265677 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:22:37.005459 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 745/1488] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:22:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:22:37.011112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77edb771-d12c-4020-8244-21be5c87fa8f None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:22:37.011556 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77edb771-d12c-4020-8244-21be5c87fa8f None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:22:37.011651 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77edb771-d12c-4020-8244-21be5c87fa8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:22:37.085263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-77edb771-d12c-4020-8244-21be5c87fa8f None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:22:37.086473 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-77edb771-d12c-4020-8244-21be5c87fa8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:22:37.154713 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:22:37.204788 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 744/1489] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:22:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:22:37.500627 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ee434-f43a-4e7e-8922-8bee0d265677 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:22:37.502197 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-3e0ee434-f43a-4e7e-8922-8bee0d265677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:22:37.542565 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:22:37.619816 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ee434-f43a-4e7e-8922-8bee0d265677 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:22:37.699165 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd7888-32ca-4cb5-9dcd-c2d364a4a31e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:22:37.699691 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd7888-32ca-4cb5-9dcd-c2d364a4a31e service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:22:37.699971 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd7888-32ca-4cb5-9dcd-c2d364a4a31e service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:22:37.700271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd7888-32ca-4cb5-9dcd-c2d364a4a31e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:22:37.701869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-12cd7888-32ca-4cb5-9dcd-c2d364a4a31e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:22:37.797029 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:22:37.827835 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd7888-32ca-4cb5-9dcd-c2d364a4a31e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:22:37.829311 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd7888-32ca-4cb5-9dcd-c2d364a4a31e service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:22:37.849790 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 746/1490] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:22:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:22:39.955564 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9537976-2fa8-432c-936f-db3e239c0298 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:22:39.956202 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9537976-2fa8-432c-936f-db3e239c0298 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:22:39.956202 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9537976-2fa8-432c-936f-db3e239c0298 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:22:39.957503 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 745/1491] 10.222.0.18 () {54 vars in 927 bytes} [Tue Aug 23 04:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:22:39.964236 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de893d-613a-45ba-b2f6-d35a82d46238 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:22:39.965709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de893d-613a-45ba-b2f6-d35a82d46238 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:22:39.966218 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de893d-613a-45ba-b2f6-d35a82d46238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:22:40.047478 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-25de893d-613a-45ba-b2f6-d35a82d46238 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:22:40.048749 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-25de893d-613a-45ba-b2f6-d35a82d46238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:22:40.149675 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 747/1492] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Aug 23 04:22:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:22:40.452184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9537976-2fa8-432c-936f-db3e239c0298 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:22:40.453845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-e9537976-2fa8-432c-936f-db3e239c0298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:22:40.571042 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9537976-2fa8-432c-936f-db3e239c0298 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:22:40.647122 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b8f1af-9a69-4f16-a684-2ac60f3861d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:22:40.647725 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8f1af-9a69-4f16-a684-2ac60f3861d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:22:40.648033 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8f1af-9a69-4f16-a684-2ac60f3861d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:22:40.648304 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8f1af-9a69-4f16-a684-2ac60f3861d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:22:40.649904 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-f6b8f1af-9a69-4f16-a684-2ac60f3861d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:22:40.659164 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:22:40.768762 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b8f1af-9a69-4f16-a684-2ac60f3861d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:22:40.770130 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b8f1af-9a69-4f16-a684-2ac60f3861d0 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:22:40.790939 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 746/1493] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:22:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:22:50.147711 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=82 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:22:50.788226 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=50 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:23:52.720780 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa433c17-ee74-4c43-912c-cdc142fe8867 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:23:52.721681 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa433c17-ee74-4c43-912c-cdc142fe8867 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:23:52.721681 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa433c17-ee74-4c43-912c-cdc142fe8867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:23:52.803797 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-fa433c17-ee74-4c43-912c-cdc142fe8867 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:23:52.805051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-fa433c17-ee74-4c43-912c-cdc142fe8867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:23:52.910524 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 748/1494] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:23:52.919398 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756be340-fae1-487e-a5d6-ed8f62dc977e None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:23:52.919940 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756be340-fae1-487e-a5d6-ed8f62dc977e None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:23:52.920075 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756be340-fae1-487e-a5d6-ed8f62dc977e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:23:53.007685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-756be340-fae1-487e-a5d6-ed8f62dc977e None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:23:53.008925 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-756be340-fae1-487e-a5d6-ed8f62dc977e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:23:53.121077 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 747/1495] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:23:53.126173 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa433c17-ee74-4c43-912c-cdc142fe8867 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:23:53.127416 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-fa433c17-ee74-4c43-912c-cdc142fe8867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:23:53.139575 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa433c17-ee74-4c43-912c-cdc142fe8867 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:23:53.205509 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:23:53.242882 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32d239-8dc3-49db-9991-c7582d78152b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:23:53.243495 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32d239-8dc3-49db-9991-c7582d78152b admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:23:53.243769 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32d239-8dc3-49db-9991-c7582d78152b admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:23:53.244076 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32d239-8dc3-49db-9991-c7582d78152b admin admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:23:53.244749 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a32d239-8dc3-49db-9991-c7582d78152b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:23:53.245046 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a32d239-8dc3-49db-9991-c7582d78152b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:23:53.246067 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: WARNING py.warnings [None req-4a32d239-8dc3-49db-9991-c7582d78152b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 23 04:23:53.246067 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: warnings.warn(msg) Aug 23 04:23:53.246067 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: Aug 23 04:23:53.246943 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a32d239-8dc3-49db-9991-c7582d78152b admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:23:53.259059 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 749/1496] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:23:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:23:53.268280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756be340-fae1-487e-a5d6-ed8f62dc977e None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:23:53.286354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756be340-fae1-487e-a5d6-ed8f62dc977e None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:23:53.490602 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:23:53.526898 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8a271-cf5b-4259-8059-572e3f3bf046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:23:53.527425 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8a271-cf5b-4259-8059-572e3f3bf046 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:23:53.527673 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8a271-cf5b-4259-8059-572e3f3bf046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:23:53.527965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8a271-cf5b-4259-8059-572e3f3bf046 admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:23:53.528658 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d8a271-cf5b-4259-8059-572e3f3bf046 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:23:53.530700 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: WARNING py.warnings [None req-b4d8a271-cf5b-4259-8059-572e3f3bf046 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 04:23:53.530700 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: warnings.warn(msg) Aug 23 04:23:53.530700 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: Aug 23 04:23:53.531958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d8a271-cf5b-4259-8059-572e3f3bf046 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:23:53.602641 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 748/1497] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:23:53 2022] POST /identity/v3/projects => generated 377 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:23:53.607255 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32d239-8dc3-49db-9991-c7582d78152b admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:23:53.608868 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-4a32d239-8dc3-49db-9991-c7582d78152b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:23:53.644292 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32d239-8dc3-49db-9991-c7582d78152b admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:23:53.745454 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82356b47-1b9e-4c6e-b65f-875bb0331c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:23:53.746188 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82356b47-1b9e-4c6e-b65f-875bb0331c6b admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:23:53.746290 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82356b47-1b9e-4c6e-b65f-875bb0331c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:23:53.746586 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82356b47-1b9e-4c6e-b65f-875bb0331c6b admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:23:53.747207 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82356b47-1b9e-4c6e-b65f-875bb0331c6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:23:53.748369 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: WARNING py.warnings [None req-82356b47-1b9e-4c6e-b65f-875bb0331c6b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 04:23:53.748369 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: warnings.warn(msg) Aug 23 04:23:53.748369 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: Aug 23 04:23:53.749579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82356b47-1b9e-4c6e-b65f-875bb0331c6b admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:23:53.793694 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 750/1498] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:23:53 2022] POST /identity/v3/users => generated 289 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:23:53.798528 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8a271-cf5b-4259-8059-572e3f3bf046 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:23:53.800935 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-b4d8a271-cf5b-4259-8059-572e3f3bf046 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:23:53.869244 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8a271-cf5b-4259-8059-572e3f3bf046 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:23:53.965702 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:23:53.979572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01e0d7-982c-4b20-ae8d-8ce3e08948ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:23:53.980225 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01e0d7-982c-4b20-ae8d-8ce3e08948ad admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:23:53.980503 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01e0d7-982c-4b20-ae8d-8ce3e08948ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:23:53.980772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01e0d7-982c-4b20-ae8d-8ce3e08948ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:23:53.981422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a01e0d7-982c-4b20-ae8d-8ce3e08948ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:23:53.983029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: WARNING py.warnings [None req-0a01e0d7-982c-4b20-ae8d-8ce3e08948ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 23 04:23:53.983029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: warnings.warn(msg) Aug 23 04:23:53.983029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: Aug 23 04:23:53.983947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a01e0d7-982c-4b20-ae8d-8ce3e08948ad admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:23:53.997409 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 749/1499] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:23:54.002104 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82356b47-1b9e-4c6e-b65f-875bb0331c6b admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:23:54.003657 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-82356b47-1b9e-4c6e-b65f-875bb0331c6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:23:54.015307 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82356b47-1b9e-4c6e-b65f-875bb0331c6b admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:23:54.040810 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:23:54.113405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d547f-21a0-48b3-950c-b0d972b87112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:23:54.113929 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d547f-21a0-48b3-950c-b0d972b87112 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:23:54.114210 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d547f-21a0-48b3-950c-b0d972b87112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:23:54.114476 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d547f-21a0-48b3-950c-b0d972b87112 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:23:54.115094 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3d547f-21a0-48b3-950c-b0d972b87112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:23:54.116183 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: WARNING py.warnings [None req-4a3d547f-21a0-48b3-950c-b0d972b87112 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 23 04:23:54.116183 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: warnings.warn(msg) Aug 23 04:23:54.116183 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: Aug 23 04:23:54.117075 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3d547f-21a0-48b3-950c-b0d972b87112 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:23:54.131337 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 751/1500] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:23:54.136756 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01e0d7-982c-4b20-ae8d-8ce3e08948ad admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:23:54.139132 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-0a01e0d7-982c-4b20-ae8d-8ce3e08948ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:23:54.174080 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01e0d7-982c-4b20-ae8d-8ce3e08948ad admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:23:54.281558 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a689871-de5f-4a1e-88e1-f495545ea62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:23:54.282229 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a689871-de5f-4a1e-88e1-f495545ea62b admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:23:54.282509 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a689871-de5f-4a1e-88e1-f495545ea62b admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:23:54.282781 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a689871-de5f-4a1e-88e1-f495545ea62b admin admin] PATH_INFO: `/v3/projects/2b63e7ccfd48434c972aada9b1b3acf8/users/45aa5e2fae5f4b7dab63c5ecbd7786da/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:23:54.307544 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a689871-de5f-4a1e-88e1-f495545ea62b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b63e7ccfd48434c972aada9b1b3acf8, user_id=45aa5e2fae5f4b7dab63c5ecbd7786da, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:23:54.308600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: WARNING py.warnings [None req-2a689871-de5f-4a1e-88e1-f495545ea62b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 04:23:54.308600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: warnings.warn(msg) Aug 23 04:23:54.308600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: Aug 23 04:23:54.310693 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a689871-de5f-4a1e-88e1-f495545ea62b admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:23:54.339588 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 750/1501] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:23:54 2022] PUT /identity/v3/projects/2b63e7ccfd48434c972aada9b1b3acf8/users/45aa5e2fae5f4b7dab63c5ecbd7786da/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:23:54.352558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d3eba-edf5-4899-8482-eacc6c94651f None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:23:54.352845 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d3eba-edf5-4899-8482-eacc6c94651f None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:23:54.353117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d3eba-edf5-4899-8482-eacc6c94651f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:23:54.412464 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-c65d3eba-edf5-4899-8482-eacc6c94651f None None] MFA Rules not processed for user `45aa5e2fae5f4b7dab63c5ecbd7786da`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:23:54.413687 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-c65d3eba-edf5-4899-8482-eacc6c94651f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:23:54.496451 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 752/1502] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:23:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:23:54.514552 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937fa71-1c56-4a16-81e6-87cd64d430b4 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:23:54.514846 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937fa71-1c56-4a16-81e6-87cd64d430b4 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:23:54.515121 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937fa71-1c56-4a16-81e6-87cd64d430b4 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:23:54.516510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 751/1503] 10.222.0.18 () {54 vars in 964 bytes} [Tue Aug 23 04:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:23:54.523309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a091ca2-d970-46f2-aaaf-b5eef7cd55c7 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:23:54.523595 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a091ca2-d970-46f2-aaaf-b5eef7cd55c7 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:23:54.523892 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a091ca2-d970-46f2-aaaf-b5eef7cd55c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:23:54.557746 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:23:54.606158 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-1a091ca2-d970-46f2-aaaf-b5eef7cd55c7 None None] MFA Rules not processed for user `2db57fe3915446acb57b4f84ca353379`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:23:54.607371 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-1a091ca2-d970-46f2-aaaf-b5eef7cd55c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:23:54.684760 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 753/1504] 10.222.0.18 () {58 vars in 1061 bytes} [Tue Aug 23 04:23:54 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:23:54.693006 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937fa71-1c56-4a16-81e6-87cd64d430b4 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:23:54.694543 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-f937fa71-1c56-4a16-81e6-87cd64d430b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:23:54.746065 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:23:54.802660 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937fa71-1c56-4a16-81e6-87cd64d430b4 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:23:54.880256 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a827d96-7b1c-4c97-b9ac-f1a021f6a413 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:23:54.880865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a827d96-7b1c-4c97-b9ac-f1a021f6a413 service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:23:54.881134 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a827d96-7b1c-4c97-b9ac-f1a021f6a413 service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:23:54.881401 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a827d96-7b1c-4c97-b9ac-f1a021f6a413 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:23:54.882991 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-4a827d96-7b1c-4c97-b9ac-f1a021f6a413 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:23:54.989263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a827d96-7b1c-4c97-b9ac-f1a021f6a413 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:23:54.990620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a827d96-7b1c-4c97-b9ac-f1a021f6a413 service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:23:54.994067 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:23:55.008828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 752/1505] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:23:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:23:55.056113 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9bd40-0809-45b9-b9cd-73012b7437d1 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:23:55.056556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9bd40-0809-45b9-b9cd-73012b7437d1 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:23:55.056668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9bd40-0809-45b9-b9cd-73012b7437d1 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:23:55.057956 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 754/1506] 10.222.0.18 () {52 vars in 830 bytes} [Tue Aug 23 04:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:23:55.062294 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a827d96-7b1c-4c97-b9ac-f1a021f6a413 service cinder] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:23:55.063531 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-4a827d96-7b1c-4c97-b9ac-f1a021f6a413 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:23:55.175017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a827d96-7b1c-4c97-b9ac-f1a021f6a413 service cinder] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:23:55.291557 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf8cb6-4ac2-4dd5-9644-b4144503ecfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:23:55.292132 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf8cb6-4ac2-4dd5-9644-b4144503ecfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:23:55.292407 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf8cb6-4ac2-4dd5-9644-b4144503ecfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:23:55.292679 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf8cb6-4ac2-4dd5-9644-b4144503ecfd admin admin] PATH_INFO: `/v3/users/45aa5e2fae5f4b7dab63c5ecbd7786da` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:23:55.305715 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf8cb6-4ac2-4dd5-9644-b4144503ecfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=45aa5e2fae5f4b7dab63c5ecbd7786da)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:23:55.306744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: WARNING py.warnings [None req-17cf8cb6-4ac2-4dd5-9644-b4144503ecfd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 04:23:55.306744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: warnings.warn(msg) Aug 23 04:23:55.306744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: Aug 23 04:23:55.307994 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf8cb6-4ac2-4dd5-9644-b4144503ecfd admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:23:55.318491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:23:55.382374 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-17cf8cb6-4ac2-4dd5-9644-b4144503ecfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45aa5e2fae5f4b7dab63c5ecbd7786da'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:23:55.382842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-17cf8cb6-4ac2-4dd5-9644-b4144503ecfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45aa5e2fae5f4b7dab63c5ecbd7786da'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:23:55.395411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-17cf8cb6-4ac2-4dd5-9644-b4144503ecfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45aa5e2fae5f4b7dab63c5ecbd7786da'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:23:55.420877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-17cf8cb6-4ac2-4dd5-9644-b4144503ecfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45aa5e2fae5f4b7dab63c5ecbd7786da'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:23:55.434489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-17cf8cb6-4ac2-4dd5-9644-b4144503ecfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45aa5e2fae5f4b7dab63c5ecbd7786da'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:23:55.440337 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 753/1507] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:23:55 2022] DELETE /identity/v3/users/45aa5e2fae5f4b7dab63c5ecbd7786da => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:23:55.446715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a87219-cbd0-439b-a433-1d190dfe4d28 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:23:55.447125 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a87219-cbd0-439b-a433-1d190dfe4d28 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:23:55.447471 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a87219-cbd0-439b-a433-1d190dfe4d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:23:55.509633 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-17a87219-cbd0-439b-a433-1d190dfe4d28 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:23:55.510859 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-17a87219-cbd0-439b-a433-1d190dfe4d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:23:55.597517 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:23:55.611011 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 755/1508] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:23:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:23:55.620901 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf8cb6-4ac2-4dd5-9644-b4144503ecfd admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:23:55.622147 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-17cf8cb6-4ac2-4dd5-9644-b4144503ecfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:23:55.751162 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf8cb6-4ac2-4dd5-9644-b4144503ecfd admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:23:55.814293 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:23:55.836231 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee185eb-1def-4db8-a380-bf01759f9db4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:23:55.836845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee185eb-1def-4db8-a380-bf01759f9db4 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:23:55.837116 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee185eb-1def-4db8-a380-bf01759f9db4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:23:55.837380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee185eb-1def-4db8-a380-bf01759f9db4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:23:55.838993 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-4ee185eb-1def-4db8-a380-bf01759f9db4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:23:55.959403 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee185eb-1def-4db8-a380-bf01759f9db4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:23:55.960890 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee185eb-1def-4db8-a380-bf01759f9db4 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:23:55.985176 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 754/1509] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:23:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:23:56.346665 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a87219-cbd0-439b-a433-1d190dfe4d28 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:23:56.347949 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-17a87219-cbd0-439b-a433-1d190dfe4d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:23:56.359541 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a87219-cbd0-439b-a433-1d190dfe4d28 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:23:56.459058 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae05a566-3151-482f-8381-9b76a8790a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:23:56.459365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05a566-3151-482f-8381-9b76a8790a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:23:56.459644 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05a566-3151-482f-8381-9b76a8790a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:23:56.459934 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05a566-3151-482f-8381-9b76a8790a8c admin admin] PATH_INFO: `/v3/projects/2b63e7ccfd48434c972aada9b1b3acf8` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:23:56.476428 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae05a566-3151-482f-8381-9b76a8790a8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b63e7ccfd48434c972aada9b1b3acf8)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:23:56.477945 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: WARNING py.warnings [None req-ae05a566-3151-482f-8381-9b76a8790a8c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 04:23:56.477945 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: warnings.warn(msg) Aug 23 04:23:56.477945 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: Aug 23 04:23:56.479778 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae05a566-3151-482f-8381-9b76a8790a8c admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:23:56.541651 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-ae05a566-3151-482f-8381-9b76a8790a8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b63e7ccfd48434c972aada9b1b3acf8', 'request_id': 'req-ae05a566-3151-482f-8381-9b76a8790a8c'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:23:56.556182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-ae05a566-3151-482f-8381-9b76a8790a8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b63e7ccfd48434c972aada9b1b3acf8', 'request_id': 'req-ae05a566-3151-482f-8381-9b76a8790a8c'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:23:56.567853 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-ae05a566-3151-482f-8381-9b76a8790a8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b63e7ccfd48434c972aada9b1b3acf8', 'request_id': 'req-ae05a566-3151-482f-8381-9b76a8790a8c'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:23:56.590256 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-ae05a566-3151-482f-8381-9b76a8790a8c admin admin] The token cache is being invalidate because project 2b63e7ccfd48434c972aada9b1b3acf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:23:56.590550 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-ae05a566-3151-482f-8381-9b76a8790a8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:23:56.590951 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-ae05a566-3151-482f-8381-9b76a8790a8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:23:56.592703 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 756/1510] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:23:56 2022] DELETE /identity/v3/projects/2b63e7ccfd48434c972aada9b1b3acf8 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:24:05.980790 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=62 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:24:06.588285 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:25:19.780146 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd19fbf-c366-490a-ac2a-aca748e0e1d4 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:25:19.780738 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd19fbf-c366-490a-ac2a-aca748e0e1d4 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:25:19.780738 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd19fbf-c366-490a-ac2a-aca748e0e1d4 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:25:19.781962 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 04:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:25:19.787878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6170f-798d-495c-9564-6c03d4666c9c None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:25:19.788661 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6170f-798d-495c-9564-6c03d4666c9c None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:25:19.788821 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6170f-798d-495c-9564-6c03d4666c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:25:19.889520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-e2a6170f-798d-495c-9564-6c03d4666c9c None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:25:19.890774 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-e2a6170f-798d-495c-9564-6c03d4666c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:25:20.010803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 04:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:25:20.165254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd19fbf-c366-490a-ac2a-aca748e0e1d4 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:25:20.167777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-2cd19fbf-c366-490a-ac2a-aca748e0e1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:25:20.296371 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd19fbf-c366-490a-ac2a-aca748e0e1d4 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:25:20.302639 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca5d81-f4cf-4530-b15b-f9c73dd447d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:25:20.303229 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca5d81-f4cf-4530-b15b-f9c73dd447d6 service nova] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:25:20.303507 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca5d81-f4cf-4530-b15b-f9c73dd447d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:25:20.303777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca5d81-f4cf-4530-b15b-f9c73dd447d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:25:20.312141 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca5d81-f4cf-4530-b15b-f9c73dd447d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:25:20.313723 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca5d81-f4cf-4530-b15b-f9c73dd447d6 service nova] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:25:20.322961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 756/1513] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:25:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:25:21.156585 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d8771-972f-4f80-9a1e-569b03617dc9 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:25:21.157502 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d8771-972f-4f80-9a1e-569b03617dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:25:21.157502 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d8771-972f-4f80-9a1e-569b03617dc9 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:25:21.159838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 758/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 04:25:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:25:21.167970 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c930853-1d7d-49eb-b773-9f0b2d48c3eb None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:25:21.168672 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c930853-1d7d-49eb-b773-9f0b2d48c3eb None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:25:21.169036 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c930853-1d7d-49eb-b773-9f0b2d48c3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:25:21.191522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-8c930853-1d7d-49eb-b773-9f0b2d48c3eb None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:25:21.192793 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-8c930853-1d7d-49eb-b773-9f0b2d48c3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:25:21.201594 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 757/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 04:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:25:21.358877 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d8771-972f-4f80-9a1e-569b03617dc9 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:25:21.378743 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d8771-972f-4f80-9a1e-569b03617dc9 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:25:21.384610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4248194-c41e-44a7-97cb-224f3cd3e7ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:25:21.385241 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4248194-c41e-44a7-97cb-224f3cd3e7ad service nova] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:25:21.385523 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4248194-c41e-44a7-97cb-224f3cd3e7ad service nova] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:25:21.385798 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4248194-c41e-44a7-97cb-224f3cd3e7ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:25:21.395079 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4248194-c41e-44a7-97cb-224f3cd3e7ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:25:21.398567 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4248194-c41e-44a7-97cb-224f3cd3e7ad service nova] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:25:21.409770 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 759/1516] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:25:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:25:21.800353 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62da28-c7b0-42b4-b765-8ddf06c9730b None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:25:21.800627 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62da28-c7b0-42b4-b765-8ddf06c9730b None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:25:21.800987 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62da28-c7b0-42b4-b765-8ddf06c9730b None None] PATH_INFO: `/v3` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:25:21.802692 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 758/1517] 10.222.0.47 () {54 vars in 925 bytes} [Tue Aug 23 04:25:21 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:25:21.810297 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef86676-76bb-4e47-a637-bccd8ba55686 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:25:21.810748 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef86676-76bb-4e47-a637-bccd8ba55686 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:25:21.811181 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef86676-76bb-4e47-a637-bccd8ba55686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:25:21.882862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-fef86676-76bb-4e47-a637-bccd8ba55686 None None] MFA Rules not processed for user `dd503fe3cf6d428e962c14ab864ee608`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:25:21.884765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-fef86676-76bb-4e47-a637-bccd8ba55686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:25:21.905869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:25:21.963122 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 760/1518] 10.222.0.47 () {58 vars in 1015 bytes} [Tue Aug 23 04:25:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:25:25.801812 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76570f3b-538f-4de0-8ab0-0b9b6d625675 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:25:25.802660 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76570f3b-538f-4de0-8ab0-0b9b6d625675 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:25:25.803348 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76570f3b-538f-4de0-8ab0-0b9b6d625675 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:25:25.807184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 759/1519] 10.222.0.18 () {54 vars in 961 bytes} [Tue Aug 23 04:25:25 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:25:25.816717 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800fce3-5e8c-45dd-8318-d279cb4c1b99 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:25:25.817212 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800fce3-5e8c-45dd-8318-d279cb4c1b99 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:25:25.817320 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800fce3-5e8c-45dd-8318-d279cb4c1b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:25:25.843318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-2800fce3-5e8c-45dd-8318-d279cb4c1b99 None None] MFA Rules not processed for user `dd503fe3cf6d428e962c14ab864ee608`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:25:25.844570 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-2800fce3-5e8c-45dd-8318-d279cb4c1b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:25:25.855940 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 761/1520] 10.222.0.18 () {58 vars in 1058 bytes} [Tue Aug 23 04:25:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:25:25.867595 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76570f3b-538f-4de0-8ab0-0b9b6d625675 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:25:25.882957 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76570f3b-538f-4de0-8ab0-0b9b6d625675 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:25:25.888562 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e379932-f645-48ab-bb64-61f5bcf38bc2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd503fe3cf6d428e962c14ab864ee608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:25:25.889148 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e379932-f645-48ab-bb64-61f5bcf38bc2 service placement] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:25:25.889428 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e379932-f645-48ab-bb64-61f5bcf38bc2 service placement] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:25:25.889692 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e379932-f645-48ab-bb64-61f5bcf38bc2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:25:25.898102 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e379932-f645-48ab-bb64-61f5bcf38bc2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:25:25.899720 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e379932-f645-48ab-bb64-61f5bcf38bc2 service placement] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:25:25.909974 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 760/1521] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:25:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:25:32.653725 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c772d26-73ae-43b0-b38f-597f8de040c4 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:25:32.654630 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c772d26-73ae-43b0-b38f-597f8de040c4 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:25:32.654630 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c772d26-73ae-43b0-b38f-597f8de040c4 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:25:32.656889 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 04:25:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:25:32.663209 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe2da2-1981-4324-b4f7-68da6bb1f957 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:25:32.663645 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe2da2-1981-4324-b4f7-68da6bb1f957 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:25:32.663769 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe2da2-1981-4324-b4f7-68da6bb1f957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:25:32.693856 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-77fe2da2-1981-4324-b4f7-68da6bb1f957 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:25:32.695070 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-77fe2da2-1981-4324-b4f7-68da6bb1f957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:25:32.704841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 04:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:25:32.858841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c772d26-73ae-43b0-b38f-597f8de040c4 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:25:32.881548 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c772d26-73ae-43b0-b38f-597f8de040c4 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:25:32.887776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45065199-3f34-4073-9cff-f8c6c69eec47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:25:32.888403 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45065199-3f34-4073-9cff-f8c6c69eec47 service nova] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:25:32.888681 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45065199-3f34-4073-9cff-f8c6c69eec47 service nova] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:25:32.888948 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45065199-3f34-4073-9cff-f8c6c69eec47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:25:32.898085 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45065199-3f34-4073-9cff-f8c6c69eec47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:25:32.899685 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45065199-3f34-4073-9cff-f8c6c69eec47 service nova] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:25:32.910018 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 763/1524] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:25:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:25:33.732990 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd90fb-4b85-4e21-844c-4e27d09b94ab None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:25:33.733518 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd90fb-4b85-4e21-844c-4e27d09b94ab None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:25:33.733518 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd90fb-4b85-4e21-844c-4e27d09b94ab None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:25:33.734815 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 04:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:25:33.740201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3a0a4-2f88-4f77-a81f-7439f8f34586 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:25:33.740609 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3a0a4-2f88-4f77-a81f-7439f8f34586 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:25:33.740773 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3a0a4-2f88-4f77-a81f-7439f8f34586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:25:33.761193 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-21a3a0a4-2f88-4f77-a81f-7439f8f34586 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:25:33.762395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-21a3a0a4-2f88-4f77-a81f-7439f8f34586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:25:33.771194 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 04:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:25:33.924708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd90fb-4b85-4e21-844c-4e27d09b94ab None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:25:33.948583 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd90fb-4b85-4e21-844c-4e27d09b94ab None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:25:33.957485 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b0a32-5bf6-4f76-9f8b-2c3cc98c75f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:25:33.958084 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b0a32-5bf6-4f76-9f8b-2c3cc98c75f2 service nova] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:25:33.958360 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b0a32-5bf6-4f76-9f8b-2c3cc98c75f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:25:33.958627 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b0a32-5bf6-4f76-9f8b-2c3cc98c75f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:25:33.970216 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493b0a32-5bf6-4f76-9f8b-2c3cc98c75f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:25:33.972911 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493b0a32-5bf6-4f76-9f8b-2c3cc98c75f2 service nova] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:25:33.988366 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 763/1527] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:25:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:25:37.768594 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ad6b7-feb3-47c5-83d4-d500d70d52ce None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:25:37.769171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ad6b7-feb3-47c5-83d4-d500d70d52ce None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:25:37.769171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ad6b7-feb3-47c5-83d4-d500d70d52ce None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:25:37.770390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 04:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:25:37.776852 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774ac39-1ab5-4064-ba46-a6a5d2ed3dca None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:25:37.777615 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774ac39-1ab5-4064-ba46-a6a5d2ed3dca None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:25:37.777790 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774ac39-1ab5-4064-ba46-a6a5d2ed3dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:25:37.800009 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-2774ac39-1ab5-4064-ba46-a6a5d2ed3dca None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:25:37.801229 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-2774ac39-1ab5-4064-ba46-a6a5d2ed3dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:25:37.810489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 04:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:25:38.096885 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ad6b7-feb3-47c5-83d4-d500d70d52ce None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:25:38.098965 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-c22ad6b7-feb3-47c5-83d4-d500d70d52ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:25:38.185767 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ad6b7-feb3-47c5-83d4-d500d70d52ce None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:25:38.192202 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e3bcb-43e0-4433-bbc9-1d38a0a02422 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:25:38.193057 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e3bcb-43e0-4433-bbc9-1d38a0a02422 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:25:38.193371 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e3bcb-43e0-4433-bbc9-1d38a0a02422 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:25:38.193714 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e3bcb-43e0-4433-bbc9-1d38a0a02422 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:25:38.207438 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4e3bcb-43e0-4433-bbc9-1d38a0a02422 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:25:38.208922 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4e3bcb-43e0-4433-bbc9-1d38a0a02422 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:25:38.218573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 766/1530] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:25:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:25:40.119869 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8abb0-5f22-4206-99a7-7b75626f30d6 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:25:40.120520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8abb0-5f22-4206-99a7-7b75626f30d6 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:25:40.120520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8abb0-5f22-4206-99a7-7b75626f30d6 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:25:40.121805 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 04:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:25:40.130394 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684d62f-06c9-4218-b1bc-2e083b2b9134 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:25:40.130837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684d62f-06c9-4218-b1bc-2e083b2b9134 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:25:40.130940 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684d62f-06c9-4218-b1bc-2e083b2b9134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:25:40.156018 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-1684d62f-06c9-4218-b1bc-2e083b2b9134 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:25:40.157202 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-1684d62f-06c9-4218-b1bc-2e083b2b9134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:25:40.166376 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 04:25:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:25:40.459259 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8abb0-5f22-4206-99a7-7b75626f30d6 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:25:40.471540 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8abb0-5f22-4206-99a7-7b75626f30d6 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:25:40.475625 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5f42d-52fa-49a0-be63-f54a31e0aa56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:25:40.476255 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5f42d-52fa-49a0-be63-f54a31e0aa56 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:25:40.476526 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5f42d-52fa-49a0-be63-f54a31e0aa56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:25:40.476791 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5f42d-52fa-49a0-be63-f54a31e0aa56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:25:40.485796 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee5f42d-52fa-49a0-be63-f54a31e0aa56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:25:40.487216 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee5f42d-52fa-49a0-be63-f54a31e0aa56 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:25:40.496265 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 766/1533] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:25:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:25:43.861823 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fb0a5-60ef-4551-8582-c81d6ff42264 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:25:43.862700 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fb0a5-60ef-4551-8582-c81d6ff42264 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:25:43.862700 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fb0a5-60ef-4551-8582-c81d6ff42264 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:25:43.864805 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 04:25:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:25:43.873338 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfa0ff-0b0c-4d54-b025-71af0f930bb0 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:25:43.873792 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfa0ff-0b0c-4d54-b025-71af0f930bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:25:43.873903 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfa0ff-0b0c-4d54-b025-71af0f930bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:25:43.896165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-4cdfa0ff-0b0c-4d54-b025-71af0f930bb0 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:25:43.897359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-4cdfa0ff-0b0c-4d54-b025-71af0f930bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:25:43.906963 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 04:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:25:44.471649 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fb0a5-60ef-4551-8582-c81d6ff42264 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:25:44.473781 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-248fb0a5-60ef-4551-8582-c81d6ff42264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:25:44.553407 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:25:44.569244 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fb0a5-60ef-4551-8582-c81d6ff42264 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:25:44.573354 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d4c473-25a0-422a-9bac-a571e246733c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:25:44.574095 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4c473-25a0-422a-9bac-a571e246733c service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:25:44.574377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4c473-25a0-422a-9bac-a571e246733c service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:25:44.574652 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4c473-25a0-422a-9bac-a571e246733c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:25:44.584011 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d4c473-25a0-422a-9bac-a571e246733c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:25:44.585472 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d4c473-25a0-422a-9bac-a571e246733c service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:25:44.595021 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 769/1536] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:25:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:25:46.499424 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47803740-348a-4a8d-a936-7028634b7126 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:25:46.500050 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47803740-348a-4a8d-a936-7028634b7126 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:25:46.500050 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47803740-348a-4a8d-a936-7028634b7126 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:25:46.501441 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 04:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:25:46.506799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b303c5e-2455-4ed3-9859-a0486fc3ce1a None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:25:46.507212 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b303c5e-2455-4ed3-9859-a0486fc3ce1a None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:25:46.507351 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b303c5e-2455-4ed3-9859-a0486fc3ce1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:25:46.531482 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-1b303c5e-2455-4ed3-9859-a0486fc3ce1a None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:25:46.534249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-1b303c5e-2455-4ed3-9859-a0486fc3ce1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:25:46.547864 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 04:25:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:25:47.092601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47803740-348a-4a8d-a936-7028634b7126 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:25:47.096095 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-47803740-348a-4a8d-a936-7028634b7126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:25:47.104884 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=84 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:25:47.112003 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47803740-348a-4a8d-a936-7028634b7126 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:25:47.115649 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18806b82-a94c-424c-a7c7-37489d48b31a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:25:47.116281 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18806b82-a94c-424c-a7c7-37489d48b31a service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:25:47.116561 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18806b82-a94c-424c-a7c7-37489d48b31a service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:25:47.116833 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18806b82-a94c-424c-a7c7-37489d48b31a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:25:47.126540 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18806b82-a94c-424c-a7c7-37489d48b31a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:25:47.128010 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18806b82-a94c-424c-a7c7-37489d48b31a service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:25:47.137588 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 769/1539] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:25:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:25:56.524154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=11 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:25:57.133481 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=5 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:09.306236 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609694bb-e651-4b07-b0aa-330f87e81ef9 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:09.306865 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609694bb-e651-4b07-b0aa-330f87e81ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:09.306865 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609694bb-e651-4b07-b0aa-330f87e81ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:09.345371 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-609694bb-e651-4b07-b0aa-330f87e81ef9 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:09.347040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-609694bb-e651-4b07-b0aa-330f87e81ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:09.361145 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 771/1540] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:09.370790 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e52f8a-7ed2-404a-b90d-5f6ebe3be93d None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:09.371437 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e52f8a-7ed2-404a-b90d-5f6ebe3be93d None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:09.371437 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e52f8a-7ed2-404a-b90d-5f6ebe3be93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:09.401042 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-90e52f8a-7ed2-404a-b90d-5f6ebe3be93d None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:09.401785 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b3e81-072f-43d7-bb74-06d94f2d0b31 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:09.402075 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b3e81-072f-43d7-bb74-06d94f2d0b31 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:09.402652 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b3e81-072f-43d7-bb74-06d94f2d0b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:09.403122 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-90e52f8a-7ed2-404a-b90d-5f6ebe3be93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:09.420077 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 770/1541] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:09.425089 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e52f8a-7ed2-404a-b90d-5f6ebe3be93d None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:09.435038 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e52f8a-7ed2-404a-b90d-5f6ebe3be93d None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:09.441345 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-ee1b3e81-072f-43d7-bb74-06d94f2d0b31 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:09.442973 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-ee1b3e81-072f-43d7-bb74-06d94f2d0b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:09.446773 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d36f59d-ae26-443c-9d92-a43c2a723bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:09.447672 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36f59d-ae26-443c-9d92-a43c2a723bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:09.448201 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36f59d-ae26-443c-9d92-a43c2a723bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:09.448683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36f59d-ae26-443c-9d92-a43c2a723bb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:09.449625 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d36f59d-ae26-443c-9d92-a43c2a723bb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:27:09.450209 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d36f59d-ae26-443c-9d92-a43c2a723bb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:09.451756 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: WARNING py.warnings [None req-6d36f59d-ae26-443c-9d92-a43c2a723bb3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 23 04:27:09.451756 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: warnings.warn(msg) Aug 23 04:27:09.451756 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: Aug 23 04:27:09.453313 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d36f59d-ae26-443c-9d92-a43c2a723bb3 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:09.473341 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 771/1542] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:27:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:27:09.473989 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 772/1543] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:09.482117 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d36f59d-ae26-443c-9d92-a43c2a723bb3 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:09.484750 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b501d2-df88-434b-8e0b-9f69fab27f86 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:09.485603 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b501d2-df88-434b-8e0b-9f69fab27f86 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:09.485901 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b501d2-df88-434b-8e0b-9f69fab27f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:09.493087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d36f59d-ae26-443c-9d92-a43c2a723bb3 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:09.504927 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f53a4-1372-4209-a0c4-32b5cb7847c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:09.505562 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f53a4-1372-4209-a0c4-32b5cb7847c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:09.505836 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f53a4-1372-4209-a0c4-32b5cb7847c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:09.506106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f53a4-1372-4209-a0c4-32b5cb7847c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:09.506738 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f53a4-1372-4209-a0c4-32b5cb7847c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:09.508752 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f53a4-1372-4209-a0c4-32b5cb7847c8 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:09.529265 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-22b501d2-df88-434b-8e0b-9f69fab27f86 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:09.530503 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-22b501d2-df88-434b-8e0b-9f69fab27f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:09.560381 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 773/1544] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:09.561418 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b7121-4035-4a58-9e5e-4d975cd9acd4 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:09.561705 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b7121-4035-4a58-9e5e-4d975cd9acd4 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:09.561977 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b7121-4035-4a58-9e5e-4d975cd9acd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:09.568667 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 772/1545] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:27:09 2022] POST /identity/v3/projects => generated 389 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:09.569705 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f53a4-1372-4209-a0c4-32b5cb7847c8 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:09.596496 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-183b7121-4035-4a58-9e5e-4d975cd9acd4 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:09.598153 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-183b7121-4035-4a58-9e5e-4d975cd9acd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:09.649009 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f53a4-1372-4209-a0c4-32b5cb7847c8 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:09.653889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d841998-b7fc-4088-8faa-a05141d03aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:09.654506 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d841998-b7fc-4088-8faa-a05141d03aa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:09.654779 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d841998-b7fc-4088-8faa-a05141d03aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:09.655048 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d841998-b7fc-4088-8faa-a05141d03aa5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:09.655689 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d841998-b7fc-4088-8faa-a05141d03aa5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:27:09.656014 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d841998-b7fc-4088-8faa-a05141d03aa5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:09.658080 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d841998-b7fc-4088-8faa-a05141d03aa5 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:09.670355 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 773/1546] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:27:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:27:09.670941 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d841998-b7fc-4088-8faa-a05141d03aa5 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:09.687401 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 774/1547] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:09.688857 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e744289-aa83-4ea9-9ef5-47a8ac271b0e None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:09.689143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e744289-aa83-4ea9-9ef5-47a8ac271b0e None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:09.689416 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e744289-aa83-4ea9-9ef5-47a8ac271b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:09.692264 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d841998-b7fc-4088-8faa-a05141d03aa5 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:09.697608 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee1375-ca35-4402-a908-f085008dc37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:09.698240 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee1375-ca35-4402-a908-f085008dc37e admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:09.698587 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee1375-ca35-4402-a908-f085008dc37e admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:09.698868 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee1375-ca35-4402-a908-f085008dc37e admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:09.699650 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee1375-ca35-4402-a908-f085008dc37e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:09.700701 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: WARNING py.warnings [None req-23ee1375-ca35-4402-a908-f085008dc37e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 04:27:09.700701 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: warnings.warn(msg) Aug 23 04:27:09.700701 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: Aug 23 04:27:09.702139 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee1375-ca35-4402-a908-f085008dc37e admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:09.710018 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-2e744289-aa83-4ea9-9ef5-47a8ac271b0e None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:09.711216 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-2e744289-aa83-4ea9-9ef5-47a8ac271b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:09.725834 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 775/1548] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:09.727484 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef549aa-083a-4628-909c-55b7139abad4 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:09.727768 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef549aa-083a-4628-909c-55b7139abad4 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:09.728062 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef549aa-083a-4628-909c-55b7139abad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:09.735538 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 774/1549] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:27:09 2022] POST /identity/v3/users => generated 295 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:09.736850 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aafef5-503f-4512-a430-5db55f5dbaec None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:09.737185 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aafef5-503f-4512-a430-5db55f5dbaec None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:09.737466 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aafef5-503f-4512-a430-5db55f5dbaec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:09.750954 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-6ef549aa-083a-4628-909c-55b7139abad4 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:09.752370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-6ef549aa-083a-4628-909c-55b7139abad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:09.757118 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-c6aafef5-503f-4512-a430-5db55f5dbaec None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:09.758288 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-c6aafef5-503f-4512-a430-5db55f5dbaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:09.760103 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 776/1550] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:09.760841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef549aa-083a-4628-909c-55b7139abad4 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:09.768185 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 775/1551] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:09.769896 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74baf442-07af-4ee8-ba32-4b3a13c79ea5 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:09.770220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74baf442-07af-4ee8-ba32-4b3a13c79ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:09.770471 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74baf442-07af-4ee8-ba32-4b3a13c79ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:09.770727 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef549aa-083a-4628-909c-55b7139abad4 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:09.775602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788e71b-1a06-4704-93c9-1808dd3045c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:09.776244 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788e71b-1a06-4704-93c9-1808dd3045c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:09.776519 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788e71b-1a06-4704-93c9-1808dd3045c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:09.776784 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788e71b-1a06-4704-93c9-1808dd3045c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:09.777399 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788e71b-1a06-4704-93c9-1808dd3045c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:09.778419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: WARNING py.warnings [None req-6788e71b-1a06-4704-93c9-1808dd3045c1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 04:27:09.778419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: warnings.warn(msg) Aug 23 04:27:09.778419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: Aug 23 04:27:09.779576 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788e71b-1a06-4704-93c9-1808dd3045c1 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:09.791064 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-74baf442-07af-4ee8-ba32-4b3a13c79ea5 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:09.792291 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-74baf442-07af-4ee8-ba32-4b3a13c79ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:09.801244 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 776/1552] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:09.802657 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35f35a-48cf-4e95-838e-8447af8e233f None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:09.802940 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35f35a-48cf-4e95-838e-8447af8e233f None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:09.803214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35f35a-48cf-4e95-838e-8447af8e233f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:09.812985 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 777/1553] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:27:09 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:09.814640 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb00ec-30ec-4b8c-b0f4-ee3ff7fd141d None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:09.814926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb00ec-30ec-4b8c-b0f4-ee3ff7fd141d None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:09.815194 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb00ec-30ec-4b8c-b0f4-ee3ff7fd141d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:09.823242 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-be35f35a-48cf-4e95-838e-8447af8e233f None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:09.824471 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-be35f35a-48cf-4e95-838e-8447af8e233f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:09.834803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-71cb00ec-30ec-4b8c-b0f4-ee3ff7fd141d None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:09.836002 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-71cb00ec-30ec-4b8c-b0f4-ee3ff7fd141d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:09.883972 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 777/1554] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:09.885302 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ce921-1310-411a-8098-8a9d95191188 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:09.885634 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ce921-1310-411a-8098-8a9d95191188 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:09.885857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ce921-1310-411a-8098-8a9d95191188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:09.896147 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 778/1555] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:09.897618 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb00ec-30ec-4b8c-b0f4-ee3ff7fd141d None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:09.907668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb00ec-30ec-4b8c-b0f4-ee3ff7fd141d None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:09.912066 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-e67ce921-1310-411a-8098-8a9d95191188 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:09.912790 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f594739-f12f-491b-bd86-79cf05eb6d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:09.913281 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-e67ce921-1310-411a-8098-8a9d95191188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:09.913412 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f594739-f12f-491b-bd86-79cf05eb6d4b admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:09.913659 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f594739-f12f-491b-bd86-79cf05eb6d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:09.913926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f594739-f12f-491b-bd86-79cf05eb6d4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:09.914713 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f594739-f12f-491b-bd86-79cf05eb6d4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:09.916394 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f594739-f12f-491b-bd86-79cf05eb6d4b admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:09.923456 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 778/1556] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:09.924767 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820bc6d5-594f-45e4-963c-75ac5ccecb32 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:09.925080 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820bc6d5-594f-45e4-963c-75ac5ccecb32 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:09.925325 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820bc6d5-594f-45e4-963c-75ac5ccecb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:09.931185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 779/1557] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:09.932747 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a409acd-fb46-4f8f-8916-dec2570f0804 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:09.933034 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a409acd-fb46-4f8f-8916-dec2570f0804 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:09.933318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a409acd-fb46-4f8f-8916-dec2570f0804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:09.940114 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:09.947062 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-820bc6d5-594f-45e4-963c-75ac5ccecb32 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:09.948290 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-820bc6d5-594f-45e4-963c-75ac5ccecb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:09.954030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-3a409acd-fb46-4f8f-8916-dec2570f0804 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:09.955222 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-3a409acd-fb46-4f8f-8916-dec2570f0804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:09.956824 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 779/1558] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:09.958002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aaa874-a261-4968-b416-72a330d843ee None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:09.958285 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aaa874-a261-4968-b416-72a330d843ee None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:09.958572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aaa874-a261-4968-b416-72a330d843ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:09.967313 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 780/1559] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:09.971352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a409acd-fb46-4f8f-8916-dec2570f0804 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:09.978103 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-77aaa874-a261-4968-b416-72a330d843ee None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:09.978235 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a409acd-fb46-4f8f-8916-dec2570f0804 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:09.979278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-77aaa874-a261-4968-b416-72a330d843ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:09.982816 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43501013-3025-4c07-a5f6-21939f66af75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:09.983414 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43501013-3025-4c07-a5f6-21939f66af75 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:09.983694 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43501013-3025-4c07-a5f6-21939f66af75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:09.983998 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43501013-3025-4c07-a5f6-21939f66af75 admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:09.984627 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43501013-3025-4c07-a5f6-21939f66af75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:09.986637 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43501013-3025-4c07-a5f6-21939f66af75 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:09.987279 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 780/1560] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:09.988068 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aaa874-a261-4968-b416-72a330d843ee None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:09.997399 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aaa874-a261-4968-b416-72a330d843ee None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:10.003101 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0539d65-9fbc-4b7b-b50b-a6a25387f924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:10.003786 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0539d65-9fbc-4b7b-b50b-a6a25387f924 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.004106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0539d65-9fbc-4b7b-b50b-a6a25387f924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.004379 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0539d65-9fbc-4b7b-b50b-a6a25387f924 admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.005031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0539d65-9fbc-4b7b-b50b-a6a25387f924 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:27:10.005335 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0539d65-9fbc-4b7b-b50b-a6a25387f924 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:10.006949 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0539d65-9fbc-4b7b-b50b-a6a25387f924 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:10.008985 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 781/1561] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:27:09 2022] POST /identity/v3/users => generated 292 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:10.008985 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05635d5c-a092-418a-b00f-7d21b88f7a7a None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.008985 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05635d5c-a092-418a-b00f-7d21b88f7a7a None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.008985 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05635d5c-a092-418a-b00f-7d21b88f7a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.027359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 781/1562] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:27:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:27:10.027359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0539d65-9fbc-4b7b-b50b-a6a25387f924 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:10.031283 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0539d65-9fbc-4b7b-b50b-a6a25387f924 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:10.031876 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-05635d5c-a092-418a-b00f-7d21b88f7a7a None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:10.033393 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-05635d5c-a092-418a-b00f-7d21b88f7a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:10.036524 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31846e-205a-4067-beb6-e2cf820e1445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:10.037156 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31846e-205a-4067-beb6-e2cf820e1445 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.037392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31846e-205a-4067-beb6-e2cf820e1445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.037660 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31846e-205a-4067-beb6-e2cf820e1445 admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.038294 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31846e-205a-4067-beb6-e2cf820e1445 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:27:10.038591 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31846e-205a-4067-beb6-e2cf820e1445 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:10.040258 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31846e-205a-4067-beb6-e2cf820e1445 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:10.041260 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 782/1563] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:10.044914 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05635d5c-a092-418a-b00f-7d21b88f7a7a None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:10.048676 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:10.052738 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05635d5c-a092-418a-b00f-7d21b88f7a7a None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:10.053385 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 782/1564] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:27:10.054198 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31846e-205a-4067-beb6-e2cf820e1445 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:10.064697 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31846e-205a-4067-beb6-e2cf820e1445 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:10.066125 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21851146-3899-4d60-8e40-e1acc732af81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:10.066980 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21851146-3899-4d60-8e40-e1acc732af81 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.067335 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21851146-3899-4d60-8e40-e1acc732af81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.067677 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21851146-3899-4d60-8e40-e1acc732af81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.068431 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21851146-3899-4d60-8e40-e1acc732af81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:10.069551 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94efe621-67a4-4142-90c8-332bb28fe16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:10.070157 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94efe621-67a4-4142-90c8-332bb28fe16c admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.070357 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21851146-3899-4d60-8e40-e1acc732af81 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:10.070470 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94efe621-67a4-4142-90c8-332bb28fe16c admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.070683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94efe621-67a4-4142-90c8-332bb28fe16c admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.071305 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94efe621-67a4-4142-90c8-332bb28fe16c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:27:10.071779 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94efe621-67a4-4142-90c8-332bb28fe16c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:10.073725 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94efe621-67a4-4142-90c8-332bb28fe16c admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:10.080803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 783/1565] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:10.081602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21851146-3899-4d60-8e40-e1acc732af81 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:10.086573 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 783/1566] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:27:10.087316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94efe621-67a4-4142-90c8-332bb28fe16c admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:10.091249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21851146-3899-4d60-8e40-e1acc732af81 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:10.098047 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf45371-714e-4fe6-8cfd-94c277e6856c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:10.098935 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf45371-714e-4fe6-8cfd-94c277e6856c admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.099102 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf45371-714e-4fe6-8cfd-94c277e6856c admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.099370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf45371-714e-4fe6-8cfd-94c277e6856c admin admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.100077 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf45371-714e-4fe6-8cfd-94c277e6856c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:27:10.100374 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf45371-714e-4fe6-8cfd-94c277e6856c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:10.101964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf45371-714e-4fe6-8cfd-94c277e6856c admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:10.103966 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94efe621-67a4-4142-90c8-332bb28fe16c admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:10.108729 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa42fe6-f8ff-422a-b74b-cb4f0bbe48c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:10.109360 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa42fe6-f8ff-422a-b74b-cb4f0bbe48c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.109672 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa42fe6-f8ff-422a-b74b-cb4f0bbe48c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.110447 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa42fe6-f8ff-422a-b74b-cb4f0bbe48c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.111163 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa42fe6-f8ff-422a-b74b-cb4f0bbe48c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:27:10.111472 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa42fe6-f8ff-422a-b74b-cb4f0bbe48c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:10.113087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa42fe6-f8ff-422a-b74b-cb4f0bbe48c0 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:10.114199 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 784/1567] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:27:10.115010 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf45371-714e-4fe6-8cfd-94c277e6856c admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:10.124675 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf45371-714e-4fe6-8cfd-94c277e6856c admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:10.125867 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 784/1568] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:27:10.126775 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa42fe6-f8ff-422a-b74b-cb4f0bbe48c0 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:10.135387 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990dda3-4bfe-4c82-9b7d-1007f875ccbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:10.136029 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990dda3-4bfe-4c82-9b7d-1007f875ccbf admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.136306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990dda3-4bfe-4c82-9b7d-1007f875ccbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.136573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990dda3-4bfe-4c82-9b7d-1007f875ccbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.136863 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa42fe6-f8ff-422a-b74b-cb4f0bbe48c0 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:10.137207 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990dda3-4bfe-4c82-9b7d-1007f875ccbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:10.138861 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990dda3-4bfe-4c82-9b7d-1007f875ccbf admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:10.141800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab166c02-4bde-4cfe-97e9-c876ea27af95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:10.142465 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab166c02-4bde-4cfe-97e9-c876ea27af95 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.142764 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab166c02-4bde-4cfe-97e9-c876ea27af95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.143058 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab166c02-4bde-4cfe-97e9-c876ea27af95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.143697 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab166c02-4bde-4cfe-97e9-c876ea27af95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:10.145719 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab166c02-4bde-4cfe-97e9-c876ea27af95 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:10.152736 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 785/1569] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:10.152736 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990dda3-4bfe-4c82-9b7d-1007f875ccbf admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:10.160946 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990dda3-4bfe-4c82-9b7d-1007f875ccbf admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:10.165695 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34193394-96ac-475d-b4ca-ddc0dfa3636a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:10.166370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34193394-96ac-475d-b4ca-ddc0dfa3636a admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.166651 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34193394-96ac-475d-b4ca-ddc0dfa3636a admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.166918 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34193394-96ac-475d-b4ca-ddc0dfa3636a admin admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.167585 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34193394-96ac-475d-b4ca-ddc0dfa3636a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:27:10.167882 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34193394-96ac-475d-b4ca-ddc0dfa3636a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:10.169794 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34193394-96ac-475d-b4ca-ddc0dfa3636a admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:10.174794 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 785/1570] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:27:10 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:10.178991 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab166c02-4bde-4cfe-97e9-c876ea27af95 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:10.189361 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 786/1571] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:27:10.189361 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34193394-96ac-475d-b4ca-ddc0dfa3636a admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:10.193436 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34193394-96ac-475d-b4ca-ddc0dfa3636a admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:10.254137 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959857b4-ad08-4602-a402-f47f7cb91e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:10.254823 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959857b4-ad08-4602-a402-f47f7cb91e7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.255080 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959857b4-ad08-4602-a402-f47f7cb91e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.255352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959857b4-ad08-4602-a402-f47f7cb91e7e admin admin] PATH_INFO: `/v3/projects/fa6589185abe4af1b3d0cba6f0699538/users/c8fa0859e5c14c53a5a2473c95efe152/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.257339 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab166c02-4bde-4cfe-97e9-c876ea27af95 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:10.262071 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ed50d-e7d9-4133-82eb-651d1f9c9850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:10.262714 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ed50d-e7d9-4133-82eb-651d1f9c9850 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.262993 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ed50d-e7d9-4133-82eb-651d1f9c9850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.263263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ed50d-e7d9-4133-82eb-651d1f9c9850 admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.263892 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ed50d-e7d9-4133-82eb-651d1f9c9850 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:10.265877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ed50d-e7d9-4133-82eb-651d1f9c9850 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:10.270104 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959857b4-ad08-4602-a402-f47f7cb91e7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa6589185abe4af1b3d0cba6f0699538, user_id=c8fa0859e5c14c53a5a2473c95efe152, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:10.271201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: WARNING py.warnings [None req-959857b4-ad08-4602-a402-f47f7cb91e7e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 04:27:10.271201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: warnings.warn(msg) Aug 23 04:27:10.271201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: Aug 23 04:27:10.273353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959857b4-ad08-4602-a402-f47f7cb91e7e admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:10.287551 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 787/1572] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:10 2022] PUT /identity/v3/projects/fa6589185abe4af1b3d0cba6f0699538/users/c8fa0859e5c14c53a5a2473c95efe152/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:10.288528 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959857b4-ad08-4602-a402-f47f7cb91e7e admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:10.294747 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 786/1573] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:27:10 2022] POST /identity/v3/projects => generated 377 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:10.295580 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ed50d-e7d9-4133-82eb-651d1f9c9850 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:10.350730 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959857b4-ad08-4602-a402-f47f7cb91e7e admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:10.355292 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ed50d-e7d9-4133-82eb-651d1f9c9850 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:10.355663 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0a7ca-2aba-4c3b-a46b-4925be599944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:10.356318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0a7ca-2aba-4c3b-a46b-4925be599944 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.356598 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0a7ca-2aba-4c3b-a46b-4925be599944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.356871 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0a7ca-2aba-4c3b-a46b-4925be599944 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.357493 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0a7ca-2aba-4c3b-a46b-4925be599944 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:10.359435 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0a7ca-2aba-4c3b-a46b-4925be599944 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:10.360220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034779d-2627-45ed-b9f1-72df22e89ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:10.360876 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034779d-2627-45ed-b9f1-72df22e89ae3 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.361159 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034779d-2627-45ed-b9f1-72df22e89ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.361433 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034779d-2627-45ed-b9f1-72df22e89ae3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.362058 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034779d-2627-45ed-b9f1-72df22e89ae3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:10.364055 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034779d-2627-45ed-b9f1-72df22e89ae3 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:10.373496 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:10.391845 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 788/1574] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:27:10 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:10.391845 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0a7ca-2aba-4c3b-a46b-4925be599944 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:10.400165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 787/1575] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:27:10 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:10.400165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034779d-2627-45ed-b9f1-72df22e89ae3 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:10.423464 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:10.465956 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034779d-2627-45ed-b9f1-72df22e89ae3 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:10.466159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0a7ca-2aba-4c3b-a46b-4925be599944 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:10.470808 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a18d90-3f55-457d-842c-ed7696a49b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:10.471000 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760fa08-c00a-4435-ba1c-aa0ffc11784f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:10.471757 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a18d90-3f55-457d-842c-ed7696a49b24 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.471971 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760fa08-c00a-4435-ba1c-aa0ffc11784f admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.472085 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a18d90-3f55-457d-842c-ed7696a49b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.472186 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760fa08-c00a-4435-ba1c-aa0ffc11784f admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.472352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a18d90-3f55-457d-842c-ed7696a49b24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.472503 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760fa08-c00a-4435-ba1c-aa0ffc11784f admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.472979 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a18d90-3f55-457d-842c-ed7696a49b24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:10.473131 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5760fa08-c00a-4435-ba1c-aa0ffc11784f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:10.474604 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5760fa08-c00a-4435-ba1c-aa0ffc11784f admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:10.474861 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a18d90-3f55-457d-842c-ed7696a49b24 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:10.485056 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 788/1576] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:10.485856 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760fa08-c00a-4435-ba1c-aa0ffc11784f admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:10.495419 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760fa08-c00a-4435-ba1c-aa0ffc11784f admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:10.500379 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b7044-c542-4029-b483-f24dd96a0b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:10.500950 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b7044-c542-4029-b483-f24dd96a0b9e admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.501229 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b7044-c542-4029-b483-f24dd96a0b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.501502 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b7044-c542-4029-b483-f24dd96a0b9e admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.502122 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b7044-c542-4029-b483-f24dd96a0b9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:10.504094 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b7044-c542-4029-b483-f24dd96a0b9e admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:10.509886 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 789/1577] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:27:10 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:10.509886 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a18d90-3f55-457d-842c-ed7696a49b24 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:10.530842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 789/1578] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:27:10 2022] POST /identity/v3/users => generated 287 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:10.532335 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab4eff-4458-488b-ad07-bf046fc0e439 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.532655 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab4eff-4458-488b-ad07-bf046fc0e439 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.532904 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab4eff-4458-488b-ad07-bf046fc0e439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.552971 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-87ab4eff-4458-488b-ad07-bf046fc0e439 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:10.554251 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-87ab4eff-4458-488b-ad07-bf046fc0e439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:10.576267 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a18d90-3f55-457d-842c-ed7696a49b24 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:10.581447 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f002f-9e7a-4c62-a488-be0728c24bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:10.582091 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f002f-9e7a-4c62-a488-be0728c24bb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.582371 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f002f-9e7a-4c62-a488-be0728c24bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.582632 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f002f-9e7a-4c62-a488-be0728c24bb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.583268 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f002f-9e7a-4c62-a488-be0728c24bb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:10.585247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f002f-9e7a-4c62-a488-be0728c24bb1 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:10.613312 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 790/1579] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:27:10 2022] POST /identity/v3/projects => generated 391 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:10.614037 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f002f-9e7a-4c62-a488-be0728c24bb1 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:10.618373 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 790/1580] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:10.618373 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab4eff-4458-488b-ad07-bf046fc0e439 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:10.629382 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab4eff-4458-488b-ad07-bf046fc0e439 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:10.678384 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f002f-9e7a-4c62-a488-be0728c24bb1 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:10.683022 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346548e5-95fb-4274-95dd-9ba01cda0ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:10.683632 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346548e5-95fb-4274-95dd-9ba01cda0ade admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.683926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346548e5-95fb-4274-95dd-9ba01cda0ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.684213 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346548e5-95fb-4274-95dd-9ba01cda0ade admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.684836 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346548e5-95fb-4274-95dd-9ba01cda0ade admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:10.686760 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346548e5-95fb-4274-95dd-9ba01cda0ade admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:10.687518 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be8307-296e-4d82-8696-d954c8a2fb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:10.688362 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be8307-296e-4d82-8696-d954c8a2fb81 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.688770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be8307-296e-4d82-8696-d954c8a2fb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.689055 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be8307-296e-4d82-8696-d954c8a2fb81 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.689675 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11be8307-296e-4d82-8696-d954c8a2fb81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:10.691595 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11be8307-296e-4d82-8696-d954c8a2fb81 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:10.708537 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:10.710352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 791/1581] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:27:10 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:10.711169 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346548e5-95fb-4274-95dd-9ba01cda0ade admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:10.716802 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 791/1582] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:27:10 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:10.717775 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be8307-296e-4d82-8696-d954c8a2fb81 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:10.722546 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346548e5-95fb-4274-95dd-9ba01cda0ade admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:10.727222 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bea3fe-3f50-4925-b23f-61c91453458c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:10.727860 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bea3fe-3f50-4925-b23f-61c91453458c admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.728165 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bea3fe-3f50-4925-b23f-61c91453458c admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.728435 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bea3fe-3f50-4925-b23f-61c91453458c admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.729057 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bea3fe-3f50-4925-b23f-61c91453458c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:10.730333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be8307-296e-4d82-8696-d954c8a2fb81 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:10.730984 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bea3fe-3f50-4925-b23f-61c91453458c admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:10.735209 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7f22f-e037-40bd-a572-dde5603248ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:10.735862 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7f22f-e037-40bd-a572-dde5603248ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.736179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7f22f-e037-40bd-a572-dde5603248ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.736447 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7f22f-e037-40bd-a572-dde5603248ba admin admin] PATH_INFO: `/v3/projects/7c99e53f92aa4ceb91b74dfeac159107/users/b85578c3e7de4c78b2e2da0000d6f332/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.752969 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 792/1583] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:27:10 2022] POST /identity/v3/users => generated 287 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:10.753800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d7f22f-e037-40bd-a572-dde5603248ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c99e53f92aa4ceb91b74dfeac159107, user_id=b85578c3e7de4c78b2e2da0000d6f332, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:10.754049 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bea3fe-3f50-4925-b23f-61c91453458c admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:10.756702 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d7f22f-e037-40bd-a572-dde5603248ba admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:10.764323 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bea3fe-3f50-4925-b23f-61c91453458c admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:10.768875 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:10.770845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 792/1584] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:10 2022] PUT /identity/v3/projects/7c99e53f92aa4ceb91b74dfeac159107/users/b85578c3e7de4c78b2e2da0000d6f332/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:10.771551 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7f22f-e037-40bd-a572-dde5603248ba admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:10.828931 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc433a-ab41-4f19-b461-1ab67c3655d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:10.829588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc433a-ab41-4f19-b461-1ab67c3655d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.829870 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc433a-ab41-4f19-b461-1ab67c3655d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.830145 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc433a-ab41-4f19-b461-1ab67c3655d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.830826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc433a-ab41-4f19-b461-1ab67c3655d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:10.832787 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc433a-ab41-4f19-b461-1ab67c3655d8 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:10.835876 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7f22f-e037-40bd-a572-dde5603248ba admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:10.840409 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195f1de2-ea3d-4e17-b10b-352353a426a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:10.841010 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f1de2-ea3d-4e17-b10b-352353a426a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.841308 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f1de2-ea3d-4e17-b10b-352353a426a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.841580 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f1de2-ea3d-4e17-b10b-352353a426a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.842217 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195f1de2-ea3d-4e17-b10b-352353a426a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:10.843789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195f1de2-ea3d-4e17-b10b-352353a426a8 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:10.854485 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 793/1585] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:10.854683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 793/1586] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:27:10 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:10.855386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195f1de2-ea3d-4e17-b10b-352353a426a8 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:10.855510 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc433a-ab41-4f19-b461-1ab67c3655d8 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:10.865436 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195f1de2-ea3d-4e17-b10b-352353a426a8 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:10.870193 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5068262-0778-4879-8352-052b820890bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:10.870789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5068262-0778-4879-8352-052b820890bb admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.871068 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5068262-0778-4879-8352-052b820890bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.871337 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5068262-0778-4879-8352-052b820890bb admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.871984 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5068262-0778-4879-8352-052b820890bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:10.873902 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5068262-0778-4879-8352-052b820890bb admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:10.895817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 794/1587] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:27:10 2022] POST /identity/v3/users => generated 296 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:10.896736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5068262-0778-4879-8352-052b820890bb admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:10.906141 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5068262-0778-4879-8352-052b820890bb admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:10.911619 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c12adc-c3ec-4497-9a40-3f51ff390bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:10.912308 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c12adc-c3ec-4497-9a40-3f51ff390bd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.912589 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c12adc-c3ec-4497-9a40-3f51ff390bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.912860 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c12adc-c3ec-4497-9a40-3f51ff390bd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.913508 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c12adc-c3ec-4497-9a40-3f51ff390bd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:10.913841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc433a-ab41-4f19-b461-1ab67c3655d8 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:10.915098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c12adc-c3ec-4497-9a40-3f51ff390bd2 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:10.917505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83bb27-5037-41ac-afb7-1f94a534c64e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:10.918103 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83bb27-5037-41ac-afb7-1f94a534c64e service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.918381 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83bb27-5037-41ac-afb7-1f94a534c64e service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.918647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83bb27-5037-41ac-afb7-1f94a534c64e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.926142 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 795/1588] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:10.926941 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c12adc-c3ec-4497-9a40-3f51ff390bd2 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:10.928137 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed83bb27-5037-41ac-afb7-1f94a534c64e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:10.930320 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed83bb27-5037-41ac-afb7-1f94a534c64e service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:10.940187 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c12adc-c3ec-4497-9a40-3f51ff390bd2 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:10.940676 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 794/1589] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:10.941546 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83bb27-5037-41ac-afb7-1f94a534c64e service neutron] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:10.946829 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caecc0d1-694e-4631-99e5-5d2faeaf9811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:10.948565 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecc0d1-694e-4631-99e5-5d2faeaf9811 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.949201 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecc0d1-694e-4631-99e5-5d2faeaf9811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.949686 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecc0d1-694e-4631-99e5-5d2faeaf9811 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.950272 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caecc0d1-694e-4631-99e5-5d2faeaf9811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:10.951322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83bb27-5037-41ac-afb7-1f94a534c64e service neutron] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:10.952183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caecc0d1-694e-4631-99e5-5d2faeaf9811 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:10.956387 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd41c08-981d-4817-a1d8-380b174f5316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:10.956994 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd41c08-981d-4817-a1d8-380b174f5316 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.957234 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd41c08-981d-4817-a1d8-380b174f5316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.957530 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd41c08-981d-4817-a1d8-380b174f5316 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.958169 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd41c08-981d-4817-a1d8-380b174f5316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:10.959754 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd41c08-981d-4817-a1d8-380b174f5316 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:10.964916 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 796/1590] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:10.966302 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf6d63-0c4d-4905-a702-be3d4128aa7a None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.966588 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf6d63-0c4d-4905-a702-be3d4128aa7a None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.966866 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf6d63-0c4d-4905-a702-be3d4128aa7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.974424 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 795/1591] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:10.975311 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd41c08-981d-4817-a1d8-380b174f5316 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:10.985851 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd41c08-981d-4817-a1d8-380b174f5316 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:10.991767 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5706867-0b79-4270-b97c-3985ae0a6419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:10.992488 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5706867-0b79-4270-b97c-3985ae0a6419 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:10.992798 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5706867-0b79-4270-b97c-3985ae0a6419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:10.993072 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5706867-0b79-4270-b97c-3985ae0a6419 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:10.994168 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5706867-0b79-4270-b97c-3985ae0a6419 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:10.995835 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5706867-0b79-4270-b97c-3985ae0a6419 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:10.997151 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-0ecf6d63-0c4d-4905-a702-be3d4128aa7a None None] MFA Rules not processed for user `b85578c3e7de4c78b2e2da0000d6f332`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:10.998407 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-0ecf6d63-0c4d-4905-a702-be3d4128aa7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:11.010208 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 796/1592] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:11.010208 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5706867-0b79-4270-b97c-3985ae0a6419 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:11.018394 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5706867-0b79-4270-b97c-3985ae0a6419 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:11.024055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8c6e8-5efa-4d81-9252-a1fb68824489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:11.024715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8c6e8-5efa-4d81-9252-a1fb68824489 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:11.025034 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8c6e8-5efa-4d81-9252-a1fb68824489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:11.025313 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8c6e8-5efa-4d81-9252-a1fb68824489 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:11.026013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd8c6e8-5efa-4d81-9252-a1fb68824489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:11.027708 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd8c6e8-5efa-4d81-9252-a1fb68824489 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:11.040017 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 797/1593] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:11.040780 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8c6e8-5efa-4d81-9252-a1fb68824489 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:11.051532 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8c6e8-5efa-4d81-9252-a1fb68824489 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:11.056618 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327cfac-6a2a-49d7-adad-348a704f6a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:11.056986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327cfac-6a2a-49d7-adad-348a704f6a52 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:11.057270 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327cfac-6a2a-49d7-adad-348a704f6a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:11.057545 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327cfac-6a2a-49d7-adad-348a704f6a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:11.058189 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327cfac-6a2a-49d7-adad-348a704f6a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:11.059818 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327cfac-6a2a-49d7-adad-348a704f6a52 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:11.070851 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 798/1594] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:11.071309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327cfac-6a2a-49d7-adad-348a704f6a52 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:11.082949 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327cfac-6a2a-49d7-adad-348a704f6a52 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:11.088550 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd3dcdc-4a01-4152-9d38-20ecabc34766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:11.089161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3dcdc-4a01-4152-9d38-20ecabc34766 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:11.089438 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3dcdc-4a01-4152-9d38-20ecabc34766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:11.089707 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3dcdc-4a01-4152-9d38-20ecabc34766 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:11.090459 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd3dcdc-4a01-4152-9d38-20ecabc34766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:11.095820 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 797/1595] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:11.095820 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf6d63-0c4d-4905-a702-be3d4128aa7a None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:11.096443 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd3dcdc-4a01-4152-9d38-20ecabc34766 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:11.105794 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf6d63-0c4d-4905-a702-be3d4128aa7a None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:11.108332 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 799/1596] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:11.109926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd3dcdc-4a01-4152-9d38-20ecabc34766 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:11.115015 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479a3ce-c9a1-4283-a1bb-11b45da2786b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:11.116530 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479a3ce-c9a1-4283-a1bb-11b45da2786b admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:11.117024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479a3ce-c9a1-4283-a1bb-11b45da2786b admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:11.117497 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479a3ce-c9a1-4283-a1bb-11b45da2786b admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:11.118580 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e479a3ce-c9a1-4283-a1bb-11b45da2786b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:11.121134 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e479a3ce-c9a1-4283-a1bb-11b45da2786b admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:11.136860 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd3dcdc-4a01-4152-9d38-20ecabc34766 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:11.140376 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 798/1597] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:11.141712 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479a3ce-c9a1-4283-a1bb-11b45da2786b admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:11.154494 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79ac5b-ff35-4ee8-ba45-c3e2bb14ccc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:11.155269 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79ac5b-ff35-4ee8-ba45-c3e2bb14ccc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:11.155544 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79ac5b-ff35-4ee8-ba45-c3e2bb14ccc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:11.156224 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79ac5b-ff35-4ee8-ba45-c3e2bb14ccc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:11.157019 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d79ac5b-ff35-4ee8-ba45-c3e2bb14ccc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:11.158462 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d79ac5b-ff35-4ee8-ba45-c3e2bb14ccc6 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:11.163084 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479a3ce-c9a1-4283-a1bb-11b45da2786b admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:11.169393 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1bd20-3ae3-4ba6-b1f6-b45f18a8fbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:11.169821 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1bd20-3ae3-4ba6-b1f6-b45f18a8fbe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:11.170108 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1bd20-3ae3-4ba6-b1f6-b45f18a8fbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:11.170384 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1bd20-3ae3-4ba6-b1f6-b45f18a8fbe0 admin admin] PATH_INFO: `/v3/projects/ac4f1d31db654ce59a76f647986ea817/users/c1565f8344aa492a90f94dd3ce8356a8/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:11.174015 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:11.177023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 800/1598] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:11.178032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79ac5b-ff35-4ee8-ba45-c3e2bb14ccc6 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:11.191160 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1bd20-3ae3-4ba6-b1f6-b45f18a8fbe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac4f1d31db654ce59a76f647986ea817, user_id=c1565f8344aa492a90f94dd3ce8356a8, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:11.192668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79ac5b-ff35-4ee8-ba45-c3e2bb14ccc6 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:11.193967 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1bd20-3ae3-4ba6-b1f6-b45f18a8fbe0 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:11.197873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a03c13-c745-4b21-bde9-f94b712cc527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:11.199342 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a03c13-c745-4b21-bde9-f94b712cc527 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:11.199718 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a03c13-c745-4b21-bde9-f94b712cc527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:11.200196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a03c13-c745-4b21-bde9-f94b712cc527 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:11.200863 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a03c13-c745-4b21-bde9-f94b712cc527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:11.202801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a03c13-c745-4b21-bde9-f94b712cc527 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:11.220976 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 799/1599] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:11 2022] PUT /identity/v3/projects/ac4f1d31db654ce59a76f647986ea817/users/c1565f8344aa492a90f94dd3ce8356a8/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:11.220976 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1bd20-3ae3-4ba6-b1f6-b45f18a8fbe0 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:11.221699 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 801/1600] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:11.221699 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a03c13-c745-4b21-bde9-f94b712cc527 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:11.234083 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a03c13-c745-4b21-bde9-f94b712cc527 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:11.257917 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:11.296091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1bd20-3ae3-4ba6-b1f6-b45f18a8fbe0 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:11.301748 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda5741e-31dc-4049-8507-6dcc2ea0f38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:11.302353 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5741e-31dc-4049-8507-6dcc2ea0f38f admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:11.302618 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5741e-31dc-4049-8507-6dcc2ea0f38f admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:11.302827 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5741e-31dc-4049-8507-6dcc2ea0f38f admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:11.303471 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda5741e-31dc-4049-8507-6dcc2ea0f38f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:11.305388 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda5741e-31dc-4049-8507-6dcc2ea0f38f admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:11.307670 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c88d50e-129b-4f4c-ae70-b2822289b430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:11.308405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88d50e-129b-4f4c-ae70-b2822289b430 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:11.309597 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88d50e-129b-4f4c-ae70-b2822289b430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:11.309882 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88d50e-129b-4f4c-ae70-b2822289b430 admin admin] PATH_INFO: `/v3/projects/3d3169d3b318421fab1d473a51637437/users/bafdb895190e45ba96e71fd348c41c3f/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:11.323927 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 800/1601] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:11.324838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda5741e-31dc-4049-8507-6dcc2ea0f38f admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:11.337454 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda5741e-31dc-4049-8507-6dcc2ea0f38f admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:11.337694 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c88d50e-129b-4f4c-ae70-b2822289b430 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d3169d3b318421fab1d473a51637437, user_id=bafdb895190e45ba96e71fd348c41c3f, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:11.341518 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c88d50e-129b-4f4c-ae70-b2822289b430 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:11.342716 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e812d-973f-4e58-b8dd-1d2771f05fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:11.343700 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e812d-973f-4e58-b8dd-1d2771f05fb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:11.343860 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e812d-973f-4e58-b8dd-1d2771f05fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:11.345094 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e812d-973f-4e58-b8dd-1d2771f05fb3 admin admin] PATH_INFO: `/v3/projects/52ab946e7d0d4a5397945bfcbf49e723/users/a91872508ecd49d98bd95651412f85a2/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:11.360004 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 802/1602] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:11 2022] PUT /identity/v3/projects/3d3169d3b318421fab1d473a51637437/users/bafdb895190e45ba96e71fd348c41c3f/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:11.361617 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95b285-29a4-4521-a510-62ddbe356733 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:11.361962 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95b285-29a4-4521-a510-62ddbe356733 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:11.362302 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95b285-29a4-4521-a510-62ddbe356733 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:11.365957 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 803/1603] 10.222.0.18 () {54 vars in 962 bytes} [Tue Aug 23 04:27:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:27:11.366841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95b285-29a4-4521-a510-62ddbe356733 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:11.372771 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e812d-973f-4e58-b8dd-1d2771f05fb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52ab946e7d0d4a5397945bfcbf49e723, user_id=a91872508ecd49d98bd95651412f85a2, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:11.377255 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e812d-973f-4e58-b8dd-1d2771f05fb3 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:11.395786 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 801/1604] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:11 2022] PUT /identity/v3/projects/52ab946e7d0d4a5397945bfcbf49e723/users/a91872508ecd49d98bd95651412f85a2/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:11.397325 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e812d-973f-4e58-b8dd-1d2771f05fb3 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:11.447507 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95b285-29a4-4521-a510-62ddbe356733 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:11.476943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e812d-973f-4e58-b8dd-1d2771f05fb3 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:11.483739 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd717f17-4b70-42e8-a414-554e25e34b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:11.484694 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd717f17-4b70-42e8-a414-554e25e34b4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:11.485253 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd717f17-4b70-42e8-a414-554e25e34b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:11.485718 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd717f17-4b70-42e8-a414-554e25e34b4a admin admin] PATH_INFO: `/v3/projects/620abd3a734b4502afa795d2cc58dadf/users/658ce95360454ea1a96a22af310e5a4f/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:11.504154 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd717f17-4b70-42e8-a414-554e25e34b4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=620abd3a734b4502afa795d2cc58dadf, user_id=658ce95360454ea1a96a22af310e5a4f, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:11.507144 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd717f17-4b70-42e8-a414-554e25e34b4a admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:11.522291 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268c648-527f-4806-bb24-8a12c6666f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:11.522965 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268c648-527f-4806-bb24-8a12c6666f81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:11.523403 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268c648-527f-4806-bb24-8a12c6666f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:11.523692 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268c648-527f-4806-bb24-8a12c6666f81 admin admin] PATH_INFO: `/v3/projects/934feb1c3d2643d3b65358816dcedf3e/users/0cda5ab380d74e3d901f6675266c4c2d/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:11.524078 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 802/1605] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:11 2022] PUT /identity/v3/projects/620abd3a734b4502afa795d2cc58dadf/users/658ce95360454ea1a96a22af310e5a4f/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:11.527873 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd2cf2-61d0-4d9f-85e8-83faf6c07e70 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:11.527873 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd2cf2-61d0-4d9f-85e8-83faf6c07e70 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:11.527873 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd2cf2-61d0-4d9f-85e8-83faf6c07e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:11.545340 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9268c648-527f-4806-bb24-8a12c6666f81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=934feb1c3d2643d3b65358816dcedf3e, user_id=0cda5ab380d74e3d901f6675266c4c2d, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:11.548384 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9268c648-527f-4806-bb24-8a12c6666f81 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:11.556508 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-bbbd2cf2-61d0-4d9f-85e8-83faf6c07e70 None None] MFA Rules not processed for user `c1565f8344aa492a90f94dd3ce8356a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:11.557731 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-bbbd2cf2-61d0-4d9f-85e8-83faf6c07e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:11.563118 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 804/1606] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:11 2022] PUT /identity/v3/projects/934feb1c3d2643d3b65358816dcedf3e/users/0cda5ab380d74e3d901f6675266c4c2d/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:11.563631 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268c648-527f-4806-bb24-8a12c6666f81 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:11.605098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:11.616091 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:11.648544 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 803/1607] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:11.650078 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086abe90-e381-4722-9491-13d368cc248f None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:11.650516 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086abe90-e381-4722-9491-13d368cc248f None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:11.651335 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086abe90-e381-4722-9491-13d368cc248f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:11.657797 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268c648-527f-4806-bb24-8a12c6666f81 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:11.664254 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b6832-aaff-4dfa-91b3-1465f8942892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:11.665046 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b6832-aaff-4dfa-91b3-1465f8942892 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:11.666102 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b6832-aaff-4dfa-91b3-1465f8942892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:11.666407 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b6832-aaff-4dfa-91b3-1465f8942892 admin admin] PATH_INFO: `/v3/projects/c113bdd75f254073abcfbd109dc7d187/users/820b9a99be574eb8b09629988c4c59c0/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:11.674544 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-086abe90-e381-4722-9491-13d368cc248f None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:11.675954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-086abe90-e381-4722-9491-13d368cc248f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:11.687427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 804/1608] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:11.690961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e0dbc-73e2-464b-a328-d9b36add2a8e None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:11.690961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e0dbc-73e2-464b-a328-d9b36add2a8e None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:11.690961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e0dbc-73e2-464b-a328-d9b36add2a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:11.698853 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b6832-aaff-4dfa-91b3-1465f8942892 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c113bdd75f254073abcfbd109dc7d187, user_id=820b9a99be574eb8b09629988c4c59c0, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:11.702672 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b6832-aaff-4dfa-91b3-1465f8942892 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:11.731893 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 805/1609] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:11 2022] PUT /identity/v3/projects/c113bdd75f254073abcfbd109dc7d187/users/820b9a99be574eb8b09629988c4c59c0/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:11.733597 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc95fc-e176-44d4-a2d9-cabcbf9a274f None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:11.733962 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc95fc-e176-44d4-a2d9-cabcbf9a274f None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:11.734328 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc95fc-e176-44d4-a2d9-cabcbf9a274f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:11.743445 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-7f5e0dbc-73e2-464b-a328-d9b36add2a8e None None] MFA Rules not processed for user `cd7b11f479a54adc8d9b8641859b07a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:11.745040 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-7f5e0dbc-73e2-464b-a328-d9b36add2a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:11.758435 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-afdc95fc-e176-44d4-a2d9-cabcbf9a274f None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:11.761092 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-afdc95fc-e176-44d4-a2d9-cabcbf9a274f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:11.841345 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 805/1610] 10.222.0.18 () {58 vars in 1059 bytes} [Tue Aug 23 04:27:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:11.842699 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f4aa8-f540-45ff-b345-9e80d761fd9e None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:11.842979 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f4aa8-f540-45ff-b345-9e80d761fd9e None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:11.843310 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 806/1611] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:11.843423 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f4aa8-f540-45ff-b345-9e80d761fd9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:11.844986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfc17f-0105-4af0-8762-f93a4352b296 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:11.845431 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfc17f-0105-4af0-8762-f93a4352b296 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:11.845835 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfc17f-0105-4af0-8762-f93a4352b296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:11.866656 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-a31f4aa8-f540-45ff-b345-9e80d761fd9e None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:11.868317 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a31f4aa8-f540-45ff-b345-9e80d761fd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:11.881047 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-efcfc17f-0105-4af0-8762-f93a4352b296 None None] MFA Rules not processed for user `0cda5ab380d74e3d901f6675266c4c2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:11.882522 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-efcfc17f-0105-4af0-8762-f93a4352b296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:11.882685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 806/1612] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:11.883551 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f4aa8-f540-45ff-b345-9e80d761fd9e None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:11.895650 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f4aa8-f540-45ff-b345-9e80d761fd9e None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:11.902538 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af3f43-a538-49a8-8a94-7163fb715da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:11.903655 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af3f43-a538-49a8-8a94-7163fb715da2 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:11.904013 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af3f43-a538-49a8-8a94-7163fb715da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:11.904502 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af3f43-a538-49a8-8a94-7163fb715da2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:11.905615 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af3f43-a538-49a8-8a94-7163fb715da2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:11.908784 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af3f43-a538-49a8-8a94-7163fb715da2 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:11.954441 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 807/1613] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:27:11 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:11.955516 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af3f43-a538-49a8-8a94-7163fb715da2 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:11.978560 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 807/1614] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:11.980055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448016d5-b3d6-4de8-86bb-a68c556e0345 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:11.980345 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448016d5-b3d6-4de8-86bb-a68c556e0345 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:11.980623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448016d5-b3d6-4de8-86bb-a68c556e0345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:12.007117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-448016d5-b3d6-4de8-86bb-a68c556e0345 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:12.008931 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-448016d5-b3d6-4de8-86bb-a68c556e0345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:12.043890 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af3f43-a538-49a8-8a94-7163fb715da2 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:12.052681 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45780ae7-8e08-4efd-9d8e-2ef50c698baf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:12.054315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45780ae7-8e08-4efd-9d8e-2ef50c698baf service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:12.054774 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45780ae7-8e08-4efd-9d8e-2ef50c698baf service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:12.055191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45780ae7-8e08-4efd-9d8e-2ef50c698baf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:12.067769 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:12.093238 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 808/1615] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:12.095047 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448016d5-b3d6-4de8-86bb-a68c556e0345 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:12.097024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:12.116910 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448016d5-b3d6-4de8-86bb-a68c556e0345 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:12.124461 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46151de7-a37f-49fb-b26e-667038d4e485 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:12.125054 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46151de7-a37f-49fb-b26e-667038d4e485 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:12.125997 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46151de7-a37f-49fb-b26e-667038d4e485 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:12.126708 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46151de7-a37f-49fb-b26e-667038d4e485 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:12.143195 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46151de7-a37f-49fb-b26e-667038d4e485 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:12.143816 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45780ae7-8e08-4efd-9d8e-2ef50c698baf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:12.145355 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45780ae7-8e08-4efd-9d8e-2ef50c698baf service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:12.146318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46151de7-a37f-49fb-b26e-667038d4e485 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:12.156659 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 808/1616] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:12.158598 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45780ae7-8e08-4efd-9d8e-2ef50c698baf service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:12.162585 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 809/1617] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:12.163691 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46151de7-a37f-49fb-b26e-667038d4e485 service neutron] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:12.178449 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46151de7-a37f-49fb-b26e-667038d4e485 service neutron] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:12.188423 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd1ffa-952b-41fc-93ae-ce03c304de0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:12.189106 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd1ffa-952b-41fc-93ae-ce03c304de0a service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:12.189513 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd1ffa-952b-41fc-93ae-ce03c304de0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:12.189786 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd1ffa-952b-41fc-93ae-ce03c304de0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:12.204963 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd1ffa-952b-41fc-93ae-ce03c304de0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:12.208664 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd1ffa-952b-41fc-93ae-ce03c304de0a service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:12.222501 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 810/1618] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:12.224117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd1ffa-952b-41fc-93ae-ce03c304de0a service neutron] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:12.237689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd1ffa-952b-41fc-93ae-ce03c304de0a service neutron] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:12.245025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf9c1c-77bc-49ba-83fd-67bf5f896643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:12.246070 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf9c1c-77bc-49ba-83fd-67bf5f896643 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:12.246515 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf9c1c-77bc-49ba-83fd-67bf5f896643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:12.246854 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf9c1c-77bc-49ba-83fd-67bf5f896643 admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:12.247811 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf9c1c-77bc-49ba-83fd-67bf5f896643 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:12.250518 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf9c1c-77bc-49ba-83fd-67bf5f896643 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:12.260699 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45780ae7-8e08-4efd-9d8e-2ef50c698baf service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:12.271179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b06591-7268-43bf-8204-0b7e75e09ecd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:12.272480 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b06591-7268-43bf-8204-0b7e75e09ecd service nova] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:12.272932 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b06591-7268-43bf-8204-0b7e75e09ecd service nova] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:12.273391 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b06591-7268-43bf-8204-0b7e75e09ecd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:12.282511 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 811/1619] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:27:12 2022] POST /identity/v3/users => generated 293 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:12.283237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467cde8-efba-48b6-9452-252e49536fb5 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:12.285556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467cde8-efba-48b6-9452-252e49536fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:12.286709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467cde8-efba-48b6-9452-252e49536fb5 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:12.290724 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 812/1620] 10.222.0.18 () {54 vars in 962 bytes} [Tue Aug 23 04:27:12 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:27:12.293461 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6467cde8-efba-48b6-9452-252e49536fb5 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:12.314379 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6467cde8-efba-48b6-9452-252e49536fb5 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:12.320616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b2ee6-7b9e-475e-ad99-f5f0ce9adc43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:12.321427 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b2ee6-7b9e-475e-ad99-f5f0ce9adc43 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:12.321788 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b2ee6-7b9e-475e-ad99-f5f0ce9adc43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:12.322147 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b2ee6-7b9e-475e-ad99-f5f0ce9adc43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:12.338389 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89b2ee6-7b9e-475e-ad99-f5f0ce9adc43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:12.340551 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89b2ee6-7b9e-475e-ad99-f5f0ce9adc43 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:12.357172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 813/1621] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:12.358229 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b2ee6-7b9e-475e-ad99-f5f0ce9adc43 service neutron] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:12.372922 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b2ee6-7b9e-475e-ad99-f5f0ce9adc43 service neutron] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:12.377995 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b06591-7268-43bf-8204-0b7e75e09ecd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:12.379942 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0373ee-6cd0-4e86-b4e0-2507c58cc31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:12.380111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b06591-7268-43bf-8204-0b7e75e09ecd service nova] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:12.380896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0373ee-6cd0-4e86-b4e0-2507c58cc31a admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:12.381250 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0373ee-6cd0-4e86-b4e0-2507c58cc31a admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:12.381594 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0373ee-6cd0-4e86-b4e0-2507c58cc31a admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:12.382365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0373ee-6cd0-4e86-b4e0-2507c58cc31a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:12.384573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0373ee-6cd0-4e86-b4e0-2507c58cc31a admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:12.404480 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 809/1622] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:12.405626 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 814/1623] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:12.408732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaceec3-27de-4640-b95b-8e1905476454 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:12.412231 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaceec3-27de-4640-b95b-8e1905476454 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:12.412641 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaceec3-27de-4640-b95b-8e1905476454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:12.413455 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0373ee-6cd0-4e86-b4e0-2507c58cc31a admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:12.442203 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-adaceec3-27de-4640-b95b-8e1905476454 None None] MFA Rules not processed for user `cd7b11f479a54adc8d9b8641859b07a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:12.443716 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-adaceec3-27de-4640-b95b-8e1905476454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:12.444197 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0373ee-6cd0-4e86-b4e0-2507c58cc31a admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:12.472412 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf87c04-7bd6-42be-8b80-da6c97249836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:12.473163 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf87c04-7bd6-42be-8b80-da6c97249836 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:12.474157 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf87c04-7bd6-42be-8b80-da6c97249836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:12.475548 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf87c04-7bd6-42be-8b80-da6c97249836 admin admin] PATH_INFO: `/v3/projects/e13cd91c29674cefbe50f35df9a4c792/users/fad8d3198d484b84a5cda3e11254f2a5/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:12.478991 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 810/1624] 10.222.0.18 () {58 vars in 1059 bytes} [Tue Aug 23 04:27:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:12.489361 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaceec3-27de-4640-b95b-8e1905476454 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:12.504302 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaceec3-27de-4640-b95b-8e1905476454 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:12.513234 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf87c04-7bd6-42be-8b80-da6c97249836 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e13cd91c29674cefbe50f35df9a4c792, user_id=fad8d3198d484b84a5cda3e11254f2a5, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:12.513685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992eae1-df0f-4f5b-8550-f27dc6a82068 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:12.514370 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992eae1-df0f-4f5b-8550-f27dc6a82068 service nova] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:12.514647 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992eae1-df0f-4f5b-8550-f27dc6a82068 service nova] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:12.514910 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992eae1-df0f-4f5b-8550-f27dc6a82068 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:12.520539 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf87c04-7bd6-42be-8b80-da6c97249836 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:12.547625 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 815/1625] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:12 2022] PUT /identity/v3/projects/e13cd91c29674cefbe50f35df9a4c792/users/fad8d3198d484b84a5cda3e11254f2a5/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:12.552301 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf87c04-7bd6-42be-8b80-da6c97249836 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:12.583797 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:12.605251 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d992eae1-df0f-4f5b-8550-f27dc6a82068 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:12.607037 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d992eae1-df0f-4f5b-8550-f27dc6a82068 service nova] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:12.630985 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 811/1626] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:12.711827 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf87c04-7bd6-42be-8b80-da6c97249836 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:12.722271 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc1f28-7f21-4d1a-af6a-932bf3c055c1 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:12.722902 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc1f28-7f21-4d1a-af6a-932bf3c055c1 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:12.723385 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc1f28-7f21-4d1a-af6a-932bf3c055c1 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:12.724277 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa20fd-9ccf-4547-b4b3-4aa051c0a333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:12.724917 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa20fd-9ccf-4547-b4b3-4aa051c0a333 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:12.725199 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa20fd-9ccf-4547-b4b3-4aa051c0a333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:12.725473 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa20fd-9ccf-4547-b4b3-4aa051c0a333 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:12.726166 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aa20fd-9ccf-4547-b4b3-4aa051c0a333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:12.727387 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 812/1627] 10.222.0.18 () {54 vars in 912 bytes} [Tue Aug 23 04:27:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:27:12.729059 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aa20fd-9ccf-4547-b4b3-4aa051c0a333 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:12.734541 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635e624-432e-4770-beae-7f3ab3ac444d None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:12.735330 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635e624-432e-4770-beae-7f3ab3ac444d None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:12.736018 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635e624-432e-4770-beae-7f3ab3ac444d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:12.759807 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 816/1628] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:12.761572 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa20fd-9ccf-4547-b4b3-4aa051c0a333 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:12.772048 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-2635e624-432e-4770-beae-7f3ab3ac444d None None] MFA Rules not processed for user `cd7b11f479a54adc8d9b8641859b07a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:12.773863 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-2635e624-432e-4770-beae-7f3ab3ac444d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:12.792682 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa20fd-9ccf-4547-b4b3-4aa051c0a333 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:12.800583 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe5948-47ca-44c2-9b46-0a4ceb634943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:12.802984 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe5948-47ca-44c2-9b46-0a4ceb634943 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:12.804842 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe5948-47ca-44c2-9b46-0a4ceb634943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:12.805428 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe5948-47ca-44c2-9b46-0a4ceb634943 admin admin] PATH_INFO: `/v3/projects/e13cd91c29674cefbe50f35df9a4c792/users/fad8d3198d484b84a5cda3e11254f2a5/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:12.811993 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe5948-47ca-44c2-9b46-0a4ceb634943 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e13cd91c29674cefbe50f35df9a4c792, user_id=fad8d3198d484b84a5cda3e11254f2a5, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:12.822694 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe5948-47ca-44c2-9b46-0a4ceb634943 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:12.856003 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 817/1629] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:12 2022] PUT /identity/v3/projects/e13cd91c29674cefbe50f35df9a4c792/users/fad8d3198d484b84a5cda3e11254f2a5/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:27:12.868893 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac256450-984b-40ee-b522-ff095f155a48 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:12.869171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac256450-984b-40ee-b522-ff095f155a48 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:12.869446 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac256450-984b-40ee-b522-ff095f155a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:12.906919 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 813/1630] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Aug 23 04:27:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:27:12.917975 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-ac256450-984b-40ee-b522-ff095f155a48 None None] MFA Rules not processed for user `fad8d3198d484b84a5cda3e11254f2a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:12.919593 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-ac256450-984b-40ee-b522-ff095f155a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:12.921842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635e624-432e-4770-beae-7f3ab3ac444d None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:12.990662 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:13.019101 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635e624-432e-4770-beae-7f3ab3ac444d None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:13.154017 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 818/1631] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:13.169909 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac256450-984b-40ee-b522-ff095f155a48 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:13.199796 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b5888-272f-4fc2-a402-307591f5345a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:13.202106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b5888-272f-4fc2-a402-307591f5345a service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:13.202106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b5888-272f-4fc2-a402-307591f5345a service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:13.203451 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b5888-272f-4fc2-a402-307591f5345a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:13.303760 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac256450-984b-40ee-b522-ff095f155a48 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:13.308354 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaebb4e-838f-40fb-9947-c6bbdd5b1d69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:13.309278 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaebb4e-838f-40fb-9947-c6bbdd5b1d69 service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:13.309487 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaebb4e-838f-40fb-9947-c6bbdd5b1d69 service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:13.309889 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaebb4e-838f-40fb-9947-c6bbdd5b1d69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:13.324857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:13.325420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaebb4e-838f-40fb-9947-c6bbdd5b1d69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:13.329947 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaebb4e-838f-40fb-9947-c6bbdd5b1d69 service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:13.352604 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 819/1632] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:27:13.354686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254430f9-e97c-4d49-a90d-b885118963a6 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:13.355267 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254430f9-e97c-4d49-a90d-b885118963a6 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:13.355786 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254430f9-e97c-4d49-a90d-b885118963a6 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:13.358567 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 820/1633] 10.222.0.18 () {54 vars in 912 bytes} [Tue Aug 23 04:27:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:27:13.365492 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca2096-4262-4ae0-8c9b-59e41e2d3a2b None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:13.366040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca2096-4262-4ae0-8c9b-59e41e2d3a2b None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:13.366386 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca2096-4262-4ae0-8c9b-59e41e2d3a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:13.388249 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b5888-272f-4fc2-a402-307591f5345a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:13.393827 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b5888-272f-4fc2-a402-307591f5345a service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:13.402702 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-4aca2096-4262-4ae0-8c9b-59e41e2d3a2b None None] MFA Rules not processed for user `cd7b11f479a54adc8d9b8641859b07a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:13.403821 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-4aca2096-4262-4ae0-8c9b-59e41e2d3a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:13.419015 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 814/1634] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:13.436947 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 821/1635] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Aug 23 04:27:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:13.474415 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b5888-272f-4fc2-a402-307591f5345a service cinder] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:13.484536 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b5888-272f-4fc2-a402-307591f5345a service cinder] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:13.490027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b6b7b2-a887-4c29-9d42-72b8b66da7a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:13.490760 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b6b7b2-a887-4c29-9d42-72b8b66da7a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:13.491162 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b6b7b2-a887-4c29-9d42-72b8b66da7a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:13.492137 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b6b7b2-a887-4c29-9d42-72b8b66da7a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:13.504915 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b6b7b2-a887-4c29-9d42-72b8b66da7a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:13.507117 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b6b7b2-a887-4c29-9d42-72b8b66da7a0 service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:13.520937 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 815/1636] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:13.904470 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca2096-4262-4ae0-8c9b-59e41e2d3a2b None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:14.086832 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca2096-4262-4ae0-8c9b-59e41e2d3a2b None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:14.101359 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080003fd-ae95-495b-a92a-2acc955fa8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:14.102029 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080003fd-ae95-495b-a92a-2acc955fa8f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:14.105999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080003fd-ae95-495b-a92a-2acc955fa8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:14.106198 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080003fd-ae95-495b-a92a-2acc955fa8f8 admin admin] PATH_INFO: `/v3/users/0cda5ab380d74e3d901f6675266c4c2d` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:14.107840 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080003fd-ae95-495b-a92a-2acc955fa8f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cda5ab380d74e3d901f6675266c4c2d)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:14.112907 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: WARNING py.warnings [None req-080003fd-ae95-495b-a92a-2acc955fa8f8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 04:27:14.112907 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: warnings.warn(msg) Aug 23 04:27:14.112907 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: Aug 23 04:27:14.115534 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080003fd-ae95-495b-a92a-2acc955fa8f8 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:14.274016 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-080003fd-ae95-495b-a92a-2acc955fa8f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cda5ab380d74e3d901f6675266c4c2d'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:14.291728 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-080003fd-ae95-495b-a92a-2acc955fa8f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cda5ab380d74e3d901f6675266c4c2d'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:14.315081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:14.373962 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-080003fd-ae95-495b-a92a-2acc955fa8f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cda5ab380d74e3d901f6675266c4c2d'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:14.376416 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-080003fd-ae95-495b-a92a-2acc955fa8f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cda5ab380d74e3d901f6675266c4c2d'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:14.407701 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-080003fd-ae95-495b-a92a-2acc955fa8f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cda5ab380d74e3d901f6675266c4c2d'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:14.442242 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 822/1637] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:27:13 2022] DELETE /identity/v3/users/0cda5ab380d74e3d901f6675266c4c2d => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:14.442963 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ea01f-f354-43bf-8a2a-008461c4567f None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:14.442963 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ea01f-f354-43bf-8a2a-008461c4567f None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:14.443617 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ea01f-f354-43bf-8a2a-008461c4567f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:14.474406 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080003fd-ae95-495b-a92a-2acc955fa8f8 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:14.482010 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-080003fd-ae95-495b-a92a-2acc955fa8f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:14.512173 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-365ea01f-f354-43bf-8a2a-008461c4567f None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:14.515210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-365ea01f-f354-43bf-8a2a-008461c4567f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:14.650109 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080003fd-ae95-495b-a92a-2acc955fa8f8 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:14.655995 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 816/1638] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:14.670176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b5867-c77e-438b-984f-0c9a5e9c3f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:14.671005 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b5867-c77e-438b-984f-0c9a5e9c3f5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:14.671291 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b5867-c77e-438b-984f-0c9a5e9c3f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:14.671569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b5867-c77e-438b-984f-0c9a5e9c3f5f admin admin] PATH_INFO: `/v3/users/c1565f8344aa492a90f94dd3ce8356a8` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:14.675374 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b5867-c77e-438b-984f-0c9a5e9c3f5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1565f8344aa492a90f94dd3ce8356a8)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:14.677383 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b5867-c77e-438b-984f-0c9a5e9c3f5f admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:14.762733 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-984b5867-c77e-438b-984f-0c9a5e9c3f5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1565f8344aa492a90f94dd3ce8356a8'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:14.774380 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-984b5867-c77e-438b-984f-0c9a5e9c3f5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1565f8344aa492a90f94dd3ce8356a8'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:14.804973 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-984b5867-c77e-438b-984f-0c9a5e9c3f5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1565f8344aa492a90f94dd3ce8356a8'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:14.805853 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-984b5867-c77e-438b-984f-0c9a5e9c3f5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1565f8344aa492a90f94dd3ce8356a8'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:14.823184 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-984b5867-c77e-438b-984f-0c9a5e9c3f5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1565f8344aa492a90f94dd3ce8356a8'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:14.830693 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ea01f-f354-43bf-8a2a-008461c4567f None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:14.833810 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-365ea01f-f354-43bf-8a2a-008461c4567f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:14.840984 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 823/1639] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:27:14 2022] DELETE /identity/v3/users/c1565f8344aa492a90f94dd3ce8356a8 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:14.843641 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0e7be-3462-4cc7-b257-26f8f547dddf None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:14.844504 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0e7be-3462-4cc7-b257-26f8f547dddf None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:14.844806 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0e7be-3462-4cc7-b257-26f8f547dddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:14.885446 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-2ef0e7be-3462-4cc7-b257-26f8f547dddf None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:14.886676 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-2ef0e7be-3462-4cc7-b257-26f8f547dddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:14.959501 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:14.975971 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 824/1640] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:14.986206 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ea01f-f354-43bf-8a2a-008461c4567f None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:14.998541 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0e7be-3462-4cc7-b257-26f8f547dddf None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:15.000712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-2ef0e7be-3462-4cc7-b257-26f8f547dddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:15.088656 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0e7be-3462-4cc7-b257-26f8f547dddf None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:15.093877 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdae059e-5307-46b7-a7f6-dacc91a2bb39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:15.094532 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae059e-5307-46b7-a7f6-dacc91a2bb39 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:15.095055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae059e-5307-46b7-a7f6-dacc91a2bb39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:15.095868 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae059e-5307-46b7-a7f6-dacc91a2bb39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:15.111235 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdae059e-5307-46b7-a7f6-dacc91a2bb39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:15.114079 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdae059e-5307-46b7-a7f6-dacc91a2bb39 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:15.130917 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 825/1641] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:15.144148 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba44cd-f9d8-4469-8aca-74d616c2ddb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:15.144827 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba44cd-f9d8-4469-8aca-74d616c2ddb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:15.145107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba44cd-f9d8-4469-8aca-74d616c2ddb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:15.145542 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba44cd-f9d8-4469-8aca-74d616c2ddb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:15.148429 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-b6ba44cd-f9d8-4469-8aca-74d616c2ddb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:15.179968 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ba44cd-f9d8-4469-8aca-74d616c2ddb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:15.181035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ba44cd-f9d8-4469-8aca-74d616c2ddb7 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:15.197972 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 817/1642] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:16.658059 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdae059e-5307-46b7-a7f6-dacc91a2bb39 service neutron] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:16.660102 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-bdae059e-5307-46b7-a7f6-dacc91a2bb39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:16.676608 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba44cd-f9d8-4469-8aca-74d616c2ddb7 service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:16.678313 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-b6ba44cd-f9d8-4469-8aca-74d616c2ddb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:16.682722 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdae059e-5307-46b7-a7f6-dacc91a2bb39 service neutron] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:16.697517 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d637a-9c07-4372-b3fa-2233bd71d0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:16.698411 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d637a-9c07-4372-b3fa-2233bd71d0eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:16.698870 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d637a-9c07-4372-b3fa-2233bd71d0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:16.699292 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d637a-9c07-4372-b3fa-2233bd71d0eb admin admin] PATH_INFO: `/v3/projects/ac4f1d31db654ce59a76f647986ea817` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:16.699881 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba44cd-f9d8-4469-8aca-74d616c2ddb7 service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:16.700608 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2d637a-9c07-4372-b3fa-2233bd71d0eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac4f1d31db654ce59a76f647986ea817)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:16.702525 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2d637a-9c07-4372-b3fa-2233bd71d0eb admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:16.715388 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abec6e-341c-49c6-90d5-746485895727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:16.716253 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abec6e-341c-49c6-90d5-746485895727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:16.716526 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abec6e-341c-49c6-90d5-746485895727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:16.716849 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abec6e-341c-49c6-90d5-746485895727 admin admin] PATH_INFO: `/v3/projects/934feb1c3d2643d3b65358816dcedf3e` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:16.720129 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9abec6e-341c-49c6-90d5-746485895727 admin admin] RBAC: Authorizing `identity:delete_project(project_id=934feb1c3d2643d3b65358816dcedf3e)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:16.721643 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: WARNING py.warnings [None req-a9abec6e-341c-49c6-90d5-746485895727 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 04:27:16.721643 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: warnings.warn(msg) Aug 23 04:27:16.721643 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: Aug 23 04:27:16.722860 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9abec6e-341c-49c6-90d5-746485895727 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:16.789062 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-db2d637a-9c07-4372-b3fa-2233bd71d0eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac4f1d31db654ce59a76f647986ea817', 'request_id': 'req-db2d637a-9c07-4372-b3fa-2233bd71d0eb'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:16.795794 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-db2d637a-9c07-4372-b3fa-2233bd71d0eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac4f1d31db654ce59a76f647986ea817', 'request_id': 'req-db2d637a-9c07-4372-b3fa-2233bd71d0eb'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:16.801531 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:16.807067 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-db2d637a-9c07-4372-b3fa-2233bd71d0eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac4f1d31db654ce59a76f647986ea817', 'request_id': 'req-db2d637a-9c07-4372-b3fa-2233bd71d0eb'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:16.837093 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-db2d637a-9c07-4372-b3fa-2233bd71d0eb admin admin] The token cache is being invalidate because project ac4f1d31db654ce59a76f647986ea817 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:27:16.837529 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-db2d637a-9c07-4372-b3fa-2233bd71d0eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:16.838404 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-db2d637a-9c07-4372-b3fa-2233bd71d0eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:16.842986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 826/1643] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:27:16 2022] DELETE /identity/v3/projects/ac4f1d31db654ce59a76f647986ea817 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:16.844167 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d637a-9c07-4372-b3fa-2233bd71d0eb admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:16.847935 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-db2d637a-9c07-4372-b3fa-2233bd71d0eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:16.880338 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a9abec6e-341c-49c6-90d5-746485895727 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '934feb1c3d2643d3b65358816dcedf3e', 'request_id': 'req-a9abec6e-341c-49c6-90d5-746485895727'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:16.911520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a9abec6e-341c-49c6-90d5-746485895727 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '934feb1c3d2643d3b65358816dcedf3e', 'request_id': 'req-a9abec6e-341c-49c6-90d5-746485895727'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:16.930233 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a9abec6e-341c-49c6-90d5-746485895727 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '934feb1c3d2643d3b65358816dcedf3e', 'request_id': 'req-a9abec6e-341c-49c6-90d5-746485895727'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:16.956981 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a9abec6e-341c-49c6-90d5-746485895727 admin admin] The token cache is being invalidate because project 934feb1c3d2643d3b65358816dcedf3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:27:16.960443 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a9abec6e-341c-49c6-90d5-746485895727 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:16.969087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a9abec6e-341c-49c6-90d5-746485895727 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:16.978147 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 818/1644] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:27:16 2022] DELETE /identity/v3/projects/934feb1c3d2643d3b65358816dcedf3e => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:16.983482 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5727b8-2720-4922-8625-4c1b93c513b8 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:16.984037 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5727b8-2720-4922-8625-4c1b93c513b8 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:16.984455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5727b8-2720-4922-8625-4c1b93c513b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:17.015429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d637a-9c07-4372-b3fa-2233bd71d0eb admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:17.036364 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-cf5727b8-2720-4922-8625-4c1b93c513b8 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:17.042544 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-cf5727b8-2720-4922-8625-4c1b93c513b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:17.095178 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:17.174021 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919be15-890a-4b89-9970-7b1f8fd0a507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:17.174021 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919be15-890a-4b89-9970-7b1f8fd0a507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:17.174294 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919be15-890a-4b89-9970-7b1f8fd0a507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:17.174525 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919be15-890a-4b89-9970-7b1f8fd0a507 admin admin] PATH_INFO: `/v3/users/fad8d3198d484b84a5cda3e11254f2a5` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:17.179753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a919be15-890a-4b89-9970-7b1f8fd0a507 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fad8d3198d484b84a5cda3e11254f2a5)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:17.183852 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a919be15-890a-4b89-9970-7b1f8fd0a507 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:17.211003 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 819/1645] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:17.220095 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab07fe0-10ba-4fbf-82b8-bc87c57f011a None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:17.220632 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab07fe0-10ba-4fbf-82b8-bc87c57f011a None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:17.220919 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab07fe0-10ba-4fbf-82b8-bc87c57f011a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:17.266041 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-aab07fe0-10ba-4fbf-82b8-bc87c57f011a None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:17.267017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-aab07fe0-10ba-4fbf-82b8-bc87c57f011a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:17.295214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 820/1646] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:27:17.299778 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab07fe0-10ba-4fbf-82b8-bc87c57f011a None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:17.301452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a919be15-890a-4b89-9970-7b1f8fd0a507 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fad8d3198d484b84a5cda3e11254f2a5'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:17.322598 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a919be15-890a-4b89-9970-7b1f8fd0a507 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fad8d3198d484b84a5cda3e11254f2a5'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:17.329721 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab07fe0-10ba-4fbf-82b8-bc87c57f011a None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:17.338141 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a65e4-1355-4c85-83d6-49ff5a0d126f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:17.341059 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a65e4-1355-4c85-83d6-49ff5a0d126f admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:17.341342 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a65e4-1355-4c85-83d6-49ff5a0d126f admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:17.341621 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a65e4-1355-4c85-83d6-49ff5a0d126f admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:17.342331 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a65e4-1355-4c85-83d6-49ff5a0d126f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:27:17.343592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a65e4-1355-4c85-83d6-49ff5a0d126f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:17.348556 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a65e4-1355-4c85-83d6-49ff5a0d126f admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:17.353595 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a919be15-890a-4b89-9970-7b1f8fd0a507 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fad8d3198d484b84a5cda3e11254f2a5'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:17.354672 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a919be15-890a-4b89-9970-7b1f8fd0a507 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fad8d3198d484b84a5cda3e11254f2a5'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:17.370128 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a919be15-890a-4b89-9970-7b1f8fd0a507 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fad8d3198d484b84a5cda3e11254f2a5'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:17.377396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 821/1647] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:27:17.380242 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a65e4-1355-4c85-83d6-49ff5a0d126f admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:17.383227 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-e14a65e4-1355-4c85-83d6-49ff5a0d126f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:17.386840 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 827/1648] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:27:16 2022] DELETE /identity/v3/users/fad8d3198d484b84a5cda3e11254f2a5 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:17.400852 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a65e4-1355-4c85-83d6-49ff5a0d126f admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:17.497208 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c0e57-ade1-4008-97ef-c8962db1f8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:17.498157 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c0e57-ade1-4008-97ef-c8962db1f8cf admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:17.498452 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c0e57-ade1-4008-97ef-c8962db1f8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:17.498884 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c0e57-ade1-4008-97ef-c8962db1f8cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:17.501813 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4c0e57-ade1-4008-97ef-c8962db1f8cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:17.504094 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4c0e57-ade1-4008-97ef-c8962db1f8cf admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:17.559419 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 822/1649] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:27:17 2022] POST /identity/v3/projects => generated 401 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:17.560080 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919be15-890a-4b89-9970-7b1f8fd0a507 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:17.591267 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919be15-890a-4b89-9970-7b1f8fd0a507 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:17.760989 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b9332-f329-43d0-a1e0-5614a8fb35f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:17.761800 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b9332-f329-43d0-a1e0-5614a8fb35f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:17.762178 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b9332-f329-43d0-a1e0-5614a8fb35f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:17.762594 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b9332-f329-43d0-a1e0-5614a8fb35f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:17.763390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b9332-f329-43d0-a1e0-5614a8fb35f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:17.767274 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b9332-f329-43d0-a1e0-5614a8fb35f7 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:17.825373 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c0e57-ade1-4008-97ef-c8962db1f8cf admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:17.830129 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 828/1650] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:27:17 2022] POST /identity/v3/users => generated 301 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:27:17.847831 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c0e57-ade1-4008-97ef-c8962db1f8cf admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:17.865551 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6feacb-1d3f-4636-ad99-fafd2a59eeca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:17.866163 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6feacb-1d3f-4636-ad99-fafd2a59eeca admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:17.866449 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6feacb-1d3f-4636-ad99-fafd2a59eeca admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:17.867805 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6feacb-1d3f-4636-ad99-fafd2a59eeca admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:17.868545 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6feacb-1d3f-4636-ad99-fafd2a59eeca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:17.877417 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6feacb-1d3f-4636-ad99-fafd2a59eeca admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:17.945869 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 823/1651] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:17.961884 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b9332-f329-43d0-a1e0-5614a8fb35f7 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:17.981678 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b9332-f329-43d0-a1e0-5614a8fb35f7 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:17.998730 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c067a3-c575-40b7-95fc-09c768fa7d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:18.000696 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c067a3-c575-40b7-95fc-09c768fa7d10 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:18.001055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c067a3-c575-40b7-95fc-09c768fa7d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:18.001344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c067a3-c575-40b7-95fc-09c768fa7d10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:18.003256 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c067a3-c575-40b7-95fc-09c768fa7d10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:18.008507 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c067a3-c575-40b7-95fc-09c768fa7d10 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:18.034884 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 829/1652] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:18.036219 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6feacb-1d3f-4636-ad99-fafd2a59eeca admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:18.043527 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:18.049655 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6feacb-1d3f-4636-ad99-fafd2a59eeca admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:18.067548 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4356d-d3bd-49f9-bcb3-6d2c264da0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:18.068691 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4356d-d3bd-49f9-bcb3-6d2c264da0d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:18.069239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4356d-d3bd-49f9-bcb3-6d2c264da0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:18.069693 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4356d-d3bd-49f9-bcb3-6d2c264da0d1 admin admin] PATH_INFO: `/v3/projects/8a770a11a4d14808b16d52629737a0ce/users/35011187d3f14a68ba26681354c8e32d/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:18.105428 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d4356d-d3bd-49f9-bcb3-6d2c264da0d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a770a11a4d14808b16d52629737a0ce, user_id=35011187d3f14a68ba26681354c8e32d, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:18.108361 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d4356d-d3bd-49f9-bcb3-6d2c264da0d1 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:18.140545 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 824/1653] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:18 2022] PUT /identity/v3/projects/8a770a11a4d14808b16d52629737a0ce/users/35011187d3f14a68ba26681354c8e32d/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:18.143352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513e665-922c-4c41-8e9b-be881a87f0f6 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:18.143352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513e665-922c-4c41-8e9b-be881a87f0f6 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:18.143608 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513e665-922c-4c41-8e9b-be881a87f0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:18.187006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-f513e665-922c-4c41-8e9b-be881a87f0f6 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:18.193781 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-f513e665-922c-4c41-8e9b-be881a87f0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:18.349527 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:18.567496 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 830/1654] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:18.597292 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4356d-d3bd-49f9-bcb3-6d2c264da0d1 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:18.602035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-04d4356d-d3bd-49f9-bcb3-6d2c264da0d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:18.767611 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4356d-d3bd-49f9-bcb3-6d2c264da0d1 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:18.775797 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d80ac-429c-47ae-b55a-76361b84d94c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:18.776494 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d80ac-429c-47ae-b55a-76361b84d94c service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:18.776772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d80ac-429c-47ae-b55a-76361b84d94c service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:18.780281 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d80ac-429c-47ae-b55a-76361b84d94c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:18.801599 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d80ac-429c-47ae-b55a-76361b84d94c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:18.803470 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d80ac-429c-47ae-b55a-76361b84d94c service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:18.828952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 825/1655] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:19.648117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f513e665-922c-4c41-8e9b-be881a87f0f6 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:19.649828 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-f513e665-922c-4c41-8e9b-be881a87f0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:19.673166 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f513e665-922c-4c41-8e9b-be881a87f0f6 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:19.691718 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5cb87-0d29-4e39-a6ee-3c67dad2f2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:19.693559 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5cb87-0d29-4e39-a6ee-3c67dad2f2b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:19.693838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5cb87-0d29-4e39-a6ee-3c67dad2f2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:19.695284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5cb87-0d29-4e39-a6ee-3c67dad2f2b9 admin admin] PATH_INFO: `/v3/projects/e13cd91c29674cefbe50f35df9a4c792` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:19.699515 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e5cb87-0d29-4e39-a6ee-3c67dad2f2b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e13cd91c29674cefbe50f35df9a4c792)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:19.703148 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e5cb87-0d29-4e39-a6ee-3c67dad2f2b9 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:19.904128 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-59e5cb87-0d29-4e39-a6ee-3c67dad2f2b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e13cd91c29674cefbe50f35df9a4c792', 'request_id': 'req-59e5cb87-0d29-4e39-a6ee-3c67dad2f2b9'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:19.944807 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-59e5cb87-0d29-4e39-a6ee-3c67dad2f2b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e13cd91c29674cefbe50f35df9a4c792', 'request_id': 'req-59e5cb87-0d29-4e39-a6ee-3c67dad2f2b9'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:19.964935 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-59e5cb87-0d29-4e39-a6ee-3c67dad2f2b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e13cd91c29674cefbe50f35df9a4c792', 'request_id': 'req-59e5cb87-0d29-4e39-a6ee-3c67dad2f2b9'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:20.010802 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-59e5cb87-0d29-4e39-a6ee-3c67dad2f2b9 admin admin] The token cache is being invalidate because project e13cd91c29674cefbe50f35df9a4c792 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:27:20.011660 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-59e5cb87-0d29-4e39-a6ee-3c67dad2f2b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:20.014970 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-59e5cb87-0d29-4e39-a6ee-3c67dad2f2b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:20.026073 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 831/1656] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:27:19 2022] DELETE /identity/v3/projects/e13cd91c29674cefbe50f35df9a4c792 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:20.031557 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646bacb-0543-444e-8f69-e1a284db8702 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:20.032733 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646bacb-0543-444e-8f69-e1a284db8702 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:20.033175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646bacb-0543-444e-8f69-e1a284db8702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:20.072859 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-c646bacb-0543-444e-8f69-e1a284db8702 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:20.076221 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-c646bacb-0543-444e-8f69-e1a284db8702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:20.218777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 826/1657] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:20.222012 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336c54f-5e3e-432b-862f-8752d326135f None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:20.222649 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336c54f-5e3e-432b-862f-8752d326135f None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:20.223102 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336c54f-5e3e-432b-862f-8752d326135f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:20.285019 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-5336c54f-5e3e-432b-862f-8752d326135f None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:20.286833 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-5336c54f-5e3e-432b-862f-8752d326135f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:20.331979 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 832/1658] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:20.336321 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646bacb-0543-444e-8f69-e1a284db8702 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:20.357651 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646bacb-0543-444e-8f69-e1a284db8702 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:20.365873 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a0270-467a-48ca-ba7e-64664c4593eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:20.366787 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a0270-467a-48ca-ba7e-64664c4593eb admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:20.367207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a0270-467a-48ca-ba7e-64664c4593eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:20.367486 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a0270-467a-48ca-ba7e-64664c4593eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:20.368393 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a0270-467a-48ca-ba7e-64664c4593eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:27:20.368696 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a0270-467a-48ca-ba7e-64664c4593eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:20.374708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a0270-467a-48ca-ba7e-64664c4593eb admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:20.407354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 827/1659] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:27:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:27:20.411792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5336c54f-5e3e-432b-862f-8752d326135f None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:20.429118 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5336c54f-5e3e-432b-862f-8752d326135f None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:20.440328 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a603d-1d9d-484f-bf72-b3674b119737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:20.441206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a603d-1d9d-484f-bf72-b3674b119737 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:20.441484 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a603d-1d9d-484f-bf72-b3674b119737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:20.441748 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a603d-1d9d-484f-bf72-b3674b119737 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:20.442820 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9a603d-1d9d-484f-bf72-b3674b119737 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:20.444362 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9a603d-1d9d-484f-bf72-b3674b119737 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:20.506620 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 833/1660] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:27:20 2022] POST /identity/v3/projects => generated 391 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:20.510944 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a0270-467a-48ca-ba7e-64664c4593eb admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:20.523040 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a0270-467a-48ca-ba7e-64664c4593eb admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:20.654871 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445ccdc-19c7-4ab4-a64b-e62ec8a1ff36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:20.655611 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445ccdc-19c7-4ab4-a64b-e62ec8a1ff36 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:20.655983 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445ccdc-19c7-4ab4-a64b-e62ec8a1ff36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:20.656336 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445ccdc-19c7-4ab4-a64b-e62ec8a1ff36 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:20.657403 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3445ccdc-19c7-4ab4-a64b-e62ec8a1ff36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:20.663839 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3445ccdc-19c7-4ab4-a64b-e62ec8a1ff36 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:20.736121 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:20.754443 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 828/1661] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:27:20 2022] POST /identity/v3/users => generated 296 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:20.758393 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a603d-1d9d-484f-bf72-b3674b119737 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:20.770767 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a603d-1d9d-484f-bf72-b3674b119737 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:20.776101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c2e15-7333-4122-bf69-731f554d83ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:20.776976 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c2e15-7333-4122-bf69-731f554d83ad admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:20.777389 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c2e15-7333-4122-bf69-731f554d83ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:20.777878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c2e15-7333-4122-bf69-731f554d83ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:20.778844 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270c2e15-7333-4122-bf69-731f554d83ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:20.782764 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270c2e15-7333-4122-bf69-731f554d83ad admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:20.811586 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 834/1662] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:20.812605 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445ccdc-19c7-4ab4-a64b-e62ec8a1ff36 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:20.837141 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445ccdc-19c7-4ab4-a64b-e62ec8a1ff36 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:20.842944 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdaa588-dd74-4a47-9afb-f4b916c4b90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:20.843791 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdaa588-dd74-4a47-9afb-f4b916c4b90f admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:20.845007 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdaa588-dd74-4a47-9afb-f4b916c4b90f admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:20.845818 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdaa588-dd74-4a47-9afb-f4b916c4b90f admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:20.847077 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdaa588-dd74-4a47-9afb-f4b916c4b90f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:20.850857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdaa588-dd74-4a47-9afb-f4b916c4b90f admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:20.887650 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 829/1663] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:27:20.889729 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c2e15-7333-4122-bf69-731f554d83ad admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:20.910610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c2e15-7333-4122-bf69-731f554d83ad admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:20.920138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272055a-b6b5-4569-938b-66e1324468f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:20.920830 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272055a-b6b5-4569-938b-66e1324468f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:20.921109 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272055a-b6b5-4569-938b-66e1324468f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:20.921378 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272055a-b6b5-4569-938b-66e1324468f6 admin admin] PATH_INFO: `/v3/projects/a85a378e0ec04d898c8b9f23eb097457/users/607f65000c0f46a48a6c6b5e16b69c46/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:20.957249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d272055a-b6b5-4569-938b-66e1324468f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a85a378e0ec04d898c8b9f23eb097457, user_id=607f65000c0f46a48a6c6b5e16b69c46, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:20.959996 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d272055a-b6b5-4569-938b-66e1324468f6 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:20.990455 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 835/1664] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:20 2022] PUT /identity/v3/projects/a85a378e0ec04d898c8b9f23eb097457/users/607f65000c0f46a48a6c6b5e16b69c46/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:21.003885 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d28c4d-80a6-4624-8c1e-78576ccb2a03 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:21.004380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d28c4d-80a6-4624-8c1e-78576ccb2a03 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:21.004937 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d28c4d-80a6-4624-8c1e-78576ccb2a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:21.115893 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-53d28c4d-80a6-4624-8c1e-78576ccb2a03 None None] MFA Rules not processed for user `607f65000c0f46a48a6c6b5e16b69c46`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:21.119096 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-53d28c4d-80a6-4624-8c1e-78576ccb2a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:21.353142 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 830/1665] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:21.357826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272055a-b6b5-4569-938b-66e1324468f6 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:21.471776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:21.498966 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272055a-b6b5-4569-938b-66e1324468f6 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:21.511033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44608bd-d919-4e84-9699-ca4a01c9714d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:21.511980 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44608bd-d919-4e84-9699-ca4a01c9714d admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:21.512614 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44608bd-d919-4e84-9699-ca4a01c9714d admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:21.513244 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44608bd-d919-4e84-9699-ca4a01c9714d admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:21.514176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44608bd-d919-4e84-9699-ca4a01c9714d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:21.518749 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44608bd-d919-4e84-9699-ca4a01c9714d admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:21.633976 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 836/1666] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:27:21 2022] POST /identity/v3/projects => generated 393 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:21.635169 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d28c4d-80a6-4624-8c1e-78576ccb2a03 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:21.663199 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d28c4d-80a6-4624-8c1e-78576ccb2a03 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:21.845301 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a064b4-937f-499d-a2cc-10bfd0207332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:21.846599 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a064b4-937f-499d-a2cc-10bfd0207332 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:21.850811 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a064b4-937f-499d-a2cc-10bfd0207332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:21.851382 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a064b4-937f-499d-a2cc-10bfd0207332 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:21.852260 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a064b4-937f-499d-a2cc-10bfd0207332 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:21.854377 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a064b4-937f-499d-a2cc-10bfd0207332 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:21.900161 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 831/1667] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:27:21 2022] POST /identity/v3/users => generated 303 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:21.901168 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44608bd-d919-4e84-9699-ca4a01c9714d admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:21.917999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44608bd-d919-4e84-9699-ca4a01c9714d admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:21.936703 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6f441-f505-4917-9aff-5f1d0f78cb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:21.937360 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6f441-f505-4917-9aff-5f1d0f78cb89 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:21.937708 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6f441-f505-4917-9aff-5f1d0f78cb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:21.938159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6f441-f505-4917-9aff-5f1d0f78cb89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:21.938887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6f441-f505-4917-9aff-5f1d0f78cb89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:21.940610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6f441-f505-4917-9aff-5f1d0f78cb89 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:21.972144 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 837/1668] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:21.973368 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a064b4-937f-499d-a2cc-10bfd0207332 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:21.991005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a064b4-937f-499d-a2cc-10bfd0207332 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:22.018768 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a775c9f4-1d82-4ea5-94e3-a7a049a30af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:22.020084 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775c9f4-1d82-4ea5-94e3-a7a049a30af0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:22.020577 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775c9f4-1d82-4ea5-94e3-a7a049a30af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:22.021104 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775c9f4-1d82-4ea5-94e3-a7a049a30af0 admin admin] PATH_INFO: `/v3/projects/62d5fe6813d34d36a11e3b7b6239b2e4/users/b1505570d28447fab698ed154e55e623/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:22.076555 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a775c9f4-1d82-4ea5-94e3-a7a049a30af0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62d5fe6813d34d36a11e3b7b6239b2e4, user_id=b1505570d28447fab698ed154e55e623, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:22.079415 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a775c9f4-1d82-4ea5-94e3-a7a049a30af0 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:22.109938 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 832/1669] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:21 2022] PUT /identity/v3/projects/62d5fe6813d34d36a11e3b7b6239b2e4/users/b1505570d28447fab698ed154e55e623/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:22.113785 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6f441-f505-4917-9aff-5f1d0f78cb89 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:22.131067 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6f441-f505-4917-9aff-5f1d0f78cb89 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:22.218453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8568b9f-cb81-427b-9e8f-736fe1fca7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:22.219149 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8568b9f-cb81-427b-9e8f-736fe1fca7f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:22.219424 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8568b9f-cb81-427b-9e8f-736fe1fca7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:22.219693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8568b9f-cb81-427b-9e8f-736fe1fca7f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:22.220561 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8568b9f-cb81-427b-9e8f-736fe1fca7f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:22.223126 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8568b9f-cb81-427b-9e8f-736fe1fca7f8 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:22.239992 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 838/1670] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:22.246802 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a775c9f4-1d82-4ea5-94e3-a7a049a30af0 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:22.301184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a775c9f4-1d82-4ea5-94e3-a7a049a30af0 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:22.309886 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff89c55-e9d8-482c-94fb-3480ee7d3b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:22.310938 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff89c55-e9d8-482c-94fb-3480ee7d3b6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:22.311441 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff89c55-e9d8-482c-94fb-3480ee7d3b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:22.312235 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff89c55-e9d8-482c-94fb-3480ee7d3b6f admin admin] PATH_INFO: `/v3/projects/62d5fe6813d34d36a11e3b7b6239b2e4/users/b1505570d28447fab698ed154e55e623/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:22.315615 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff89c55-e9d8-482c-94fb-3480ee7d3b6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=62d5fe6813d34d36a11e3b7b6239b2e4, user_id=b1505570d28447fab698ed154e55e623, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:22.323214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff89c55-e9d8-482c-94fb-3480ee7d3b6f admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:22.342430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 833/1671] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:22 2022] PUT /identity/v3/projects/62d5fe6813d34d36a11e3b7b6239b2e4/users/b1505570d28447fab698ed154e55e623/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:22.353513 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37dceb5-7559-4d1a-a44f-35879b98d383 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:22.353802 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37dceb5-7559-4d1a-a44f-35879b98d383 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:22.354070 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37dceb5-7559-4d1a-a44f-35879b98d383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:22.428569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-f37dceb5-7559-4d1a-a44f-35879b98d383 None None] MFA Rules not processed for user `b1505570d28447fab698ed154e55e623`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:22.430153 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-f37dceb5-7559-4d1a-a44f-35879b98d383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:22.590256 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 839/1672] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:22 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:22.598275 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff89c55-e9d8-482c-94fb-3480ee7d3b6f admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:22.600467 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-eff89c55-e9d8-482c-94fb-3480ee7d3b6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:22.693347 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:22.781964 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff89c55-e9d8-482c-94fb-3480ee7d3b6f admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:22.799210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37a13f-6da6-449a-b957-a2c644d5f0b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:22.799878 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37a13f-6da6-449a-b957-a2c644d5f0b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:22.800458 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37a13f-6da6-449a-b957-a2c644d5f0b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:22.800742 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37a13f-6da6-449a-b957-a2c644d5f0b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:22.836832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37a13f-6da6-449a-b957-a2c644d5f0b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:22.838624 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37a13f-6da6-449a-b957-a2c644d5f0b5 service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:22.863090 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37dceb5-7559-4d1a-a44f-35879b98d383 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:22.863815 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-f37dceb5-7559-4d1a-a44f-35879b98d383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:22.889165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 834/1673] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:22 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:22.972508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:23.010253 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37dceb5-7559-4d1a-a44f-35879b98d383 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:23.030173 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b22b2c-5009-4a4e-91aa-1de3bc2192cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:23.030874 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b22b2c-5009-4a4e-91aa-1de3bc2192cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:23.031285 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b22b2c-5009-4a4e-91aa-1de3bc2192cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:23.032944 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b22b2c-5009-4a4e-91aa-1de3bc2192cb admin admin] PATH_INFO: `/v3/users/b85578c3e7de4c78b2e2da0000d6f332` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:23.038730 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b22b2c-5009-4a4e-91aa-1de3bc2192cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b85578c3e7de4c78b2e2da0000d6f332)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:23.041705 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b22b2c-5009-4a4e-91aa-1de3bc2192cb admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:23.066124 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37a13f-6da6-449a-b957-a2c644d5f0b5 service cinder] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:23.067659 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-db37a13f-6da6-449a-b957-a2c644d5f0b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:23.081729 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37a13f-6da6-449a-b957-a2c644d5f0b5 service cinder] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:23.088290 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9857f-b64b-4993-baba-e77f3af4415d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:23.089557 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9857f-b64b-4993-baba-e77f3af4415d service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:23.090024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9857f-b64b-4993-baba-e77f3af4415d service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:23.090478 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9857f-b64b-4993-baba-e77f3af4415d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:23.188607 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-98b22b2c-5009-4a4e-91aa-1de3bc2192cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b85578c3e7de4c78b2e2da0000d6f332'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:23.194440 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9857f-b64b-4993-baba-e77f3af4415d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:23.196881 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9857f-b64b-4993-baba-e77f3af4415d service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:23.202448 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-98b22b2c-5009-4a4e-91aa-1de3bc2192cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b85578c3e7de4c78b2e2da0000d6f332'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:23.213645 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 835/1674] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:23 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:23.216164 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ece43-55ba-4504-83c8-a7d3973404f7 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:23.216733 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ece43-55ba-4504-83c8-a7d3973404f7 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:23.217111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ece43-55ba-4504-83c8-a7d3973404f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:23.233021 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-98b22b2c-5009-4a4e-91aa-1de3bc2192cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b85578c3e7de4c78b2e2da0000d6f332'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:23.234621 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-98b22b2c-5009-4a4e-91aa-1de3bc2192cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b85578c3e7de4c78b2e2da0000d6f332'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:23.249647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-98b22b2c-5009-4a4e-91aa-1de3bc2192cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b85578c3e7de4c78b2e2da0000d6f332'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:23.262240 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 840/1675] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:27:22 2022] DELETE /identity/v3/users/b85578c3e7de4c78b2e2da0000d6f332 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:27:23.266792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eef41b-9cb0-48f0-add6-fca954ca2521 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:23.267077 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eef41b-9cb0-48f0-add6-fca954ca2521 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:23.267608 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eef41b-9cb0-48f0-add6-fca954ca2521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:23.278475 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-ca3ece43-55ba-4504-83c8-a7d3973404f7 None None] MFA Rules not processed for user `35011187d3f14a68ba26681354c8e32d`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:23.285307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-ca3ece43-55ba-4504-83c8-a7d3973404f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:23.318994 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-b1eef41b-9cb0-48f0-add6-fca954ca2521 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:23.320306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-b1eef41b-9cb0-48f0-add6-fca954ca2521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:23.489402 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 836/1676] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:23 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:23.490712 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ece43-55ba-4504-83c8-a7d3973404f7 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:23.494741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-ca3ece43-55ba-4504-83c8-a7d3973404f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:23.526944 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 841/1677] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:27:23.564472 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eef41b-9cb0-48f0-add6-fca954ca2521 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:23.566012 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-b1eef41b-9cb0-48f0-add6-fca954ca2521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:23.725054 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ece43-55ba-4504-83c8-a7d3973404f7 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:23.744839 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ceead3-5efc-4454-8c6a-88b810183b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:23.745501 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ceead3-5efc-4454-8c6a-88b810183b1e admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:23.745786 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ceead3-5efc-4454-8c6a-88b810183b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:23.746061 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ceead3-5efc-4454-8c6a-88b810183b1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:23.746703 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ceead3-5efc-4454-8c6a-88b810183b1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:23.748693 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ceead3-5efc-4454-8c6a-88b810183b1e admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:23.764875 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eef41b-9cb0-48f0-add6-fca954ca2521 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:23.765279 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:23.801252 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9bedc0-c474-497d-97b1-eb7bf7321e46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:23.803266 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9bedc0-c474-497d-97b1-eb7bf7321e46 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:23.805058 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9bedc0-c474-497d-97b1-eb7bf7321e46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:23.806508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9bedc0-c474-497d-97b1-eb7bf7321e46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:23.825310 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 837/1678] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:27:23 2022] POST /identity/v3/projects => generated 403 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:27:23.829491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ceead3-5efc-4454-8c6a-88b810183b1e admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:23.843343 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9bedc0-c474-497d-97b1-eb7bf7321e46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:23.850661 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9bedc0-c474-497d-97b1-eb7bf7321e46 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:23.892700 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 842/1679] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:27:24.000663 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ceead3-5efc-4454-8c6a-88b810183b1e admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:24.011424 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74384a-faeb-4478-990b-6a662c2817ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:24.012203 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74384a-faeb-4478-990b-6a662c2817ab admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:24.012755 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74384a-faeb-4478-990b-6a662c2817ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:24.013082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74384a-faeb-4478-990b-6a662c2817ab admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:24.013802 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e74384a-faeb-4478-990b-6a662c2817ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:24.016645 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e74384a-faeb-4478-990b-6a662c2817ab admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:24.057293 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 838/1680] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:27:23 2022] POST /identity/v3/users => generated 308 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:24.061104 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9bedc0-c474-497d-97b1-eb7bf7321e46 service neutron] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:24.074948 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9bedc0-c474-497d-97b1-eb7bf7321e46 service neutron] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:24.087593 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4679f0-61d9-4be6-bbe8-c2d663eee90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:24.088253 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4679f0-61d9-4be6-bbe8-c2d663eee90b admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:24.088534 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4679f0-61d9-4be6-bbe8-c2d663eee90b admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:24.088809 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4679f0-61d9-4be6-bbe8-c2d663eee90b admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:24.090946 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4679f0-61d9-4be6-bbe8-c2d663eee90b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:24.094379 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4679f0-61d9-4be6-bbe8-c2d663eee90b admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:24.100715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:24.129134 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 843/1681] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:24.132954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74384a-faeb-4478-990b-6a662c2817ab admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:24.148639 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74384a-faeb-4478-990b-6a662c2817ab admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:24.157643 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687da2c1-c79b-440c-94c9-b6f5cf559fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:24.159073 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687da2c1-c79b-440c-94c9-b6f5cf559fb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:24.159824 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687da2c1-c79b-440c-94c9-b6f5cf559fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:24.161579 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687da2c1-c79b-440c-94c9-b6f5cf559fb7 admin admin] PATH_INFO: `/v3/projects/afa34b8e375c470f97ee04ad9ec271ba/users/61739a2a9d474545a93a81d5a0c31aeb/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:24.205677 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687da2c1-c79b-440c-94c9-b6f5cf559fb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afa34b8e375c470f97ee04ad9ec271ba, user_id=61739a2a9d474545a93a81d5a0c31aeb, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:24.211442 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687da2c1-c79b-440c-94c9-b6f5cf559fb7 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:24.241774 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 839/1682] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:24 2022] PUT /identity/v3/projects/afa34b8e375c470f97ee04ad9ec271ba/users/61739a2a9d474545a93a81d5a0c31aeb/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:24.246291 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4679f0-61d9-4be6-bbe8-c2d663eee90b admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:24.256811 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4679f0-61d9-4be6-bbe8-c2d663eee90b admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:24.336145 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75958439-d87f-4fb6-8e19-37497434f990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:24.338602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75958439-d87f-4fb6-8e19-37497434f990 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:24.338965 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75958439-d87f-4fb6-8e19-37497434f990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:24.339301 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75958439-d87f-4fb6-8e19-37497434f990 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:24.340113 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75958439-d87f-4fb6-8e19-37497434f990 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:24.342136 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75958439-d87f-4fb6-8e19-37497434f990 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:24.358452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 844/1683] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:24.362570 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687da2c1-c79b-440c-94c9-b6f5cf559fb7 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:24.379985 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687da2c1-c79b-440c-94c9-b6f5cf559fb7 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:24.391989 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be9bf7-661e-4ff2-8124-c3b91c32693a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:24.392866 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be9bf7-661e-4ff2-8124-c3b91c32693a admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:24.392866 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be9bf7-661e-4ff2-8124-c3b91c32693a admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:24.393085 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be9bf7-661e-4ff2-8124-c3b91c32693a admin admin] PATH_INFO: `/v3/projects/afa34b8e375c470f97ee04ad9ec271ba/users/61739a2a9d474545a93a81d5a0c31aeb/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:24.397550 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be9bf7-661e-4ff2-8124-c3b91c32693a admin admin] RBAC: Authorizing `identity:create_grant(project_id=afa34b8e375c470f97ee04ad9ec271ba, user_id=61739a2a9d474545a93a81d5a0c31aeb, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:24.402823 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be9bf7-661e-4ff2-8124-c3b91c32693a admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:24.444173 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 840/1684] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:24 2022] PUT /identity/v3/projects/afa34b8e375c470f97ee04ad9ec271ba/users/61739a2a9d474545a93a81d5a0c31aeb/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:24.884534 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4947d6-8ed7-438d-ba95-c7800e01b4af None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:24.884854 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4947d6-8ed7-438d-ba95-c7800e01b4af None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:24.885441 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4947d6-8ed7-438d-ba95-c7800e01b4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:24.945122 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-3e4947d6-8ed7-438d-ba95-c7800e01b4af None None] MFA Rules not processed for user `c8fa0859e5c14c53a5a2473c95efe152`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:24.947663 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-3e4947d6-8ed7-438d-ba95-c7800e01b4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:25.155653 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 845/1685] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:24 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:25.157590 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be9bf7-661e-4ff2-8124-c3b91c32693a admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:25.159528 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-f9be9bf7-661e-4ff2-8124-c3b91c32693a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:25.336523 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be9bf7-661e-4ff2-8124-c3b91c32693a admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:25.348502 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3703db-c21b-4b34-abe8-a346626ce14b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:25.349175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3703db-c21b-4b34-abe8-a346626ce14b service nova] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:25.349505 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3703db-c21b-4b34-abe8-a346626ce14b service nova] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:25.349807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3703db-c21b-4b34-abe8-a346626ce14b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:25.380248 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3703db-c21b-4b34-abe8-a346626ce14b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:25.381902 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3703db-c21b-4b34-abe8-a346626ce14b service nova] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:25.405729 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 841/1686] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:25 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:27.311429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4947d6-8ed7-438d-ba95-c7800e01b4af None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:27.317108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-3e4947d6-8ed7-438d-ba95-c7800e01b4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:27.431147 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9503532-d08e-4a45-801e-3c2d006313ba None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:27.432529 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9503532-d08e-4a45-801e-3c2d006313ba None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:27.432980 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9503532-d08e-4a45-801e-3c2d006313ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:27.501280 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4947d6-8ed7-438d-ba95-c7800e01b4af None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:27.515100 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-a9503532-d08e-4a45-801e-3c2d006313ba None None] MFA Rules not processed for user `61739a2a9d474545a93a81d5a0c31aeb`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:27.515797 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c9d8ee-b1be-4435-9fc4-3c0cc126a7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:27.516756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9d8ee-b1be-4435-9fc4-3c0cc126a7f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:27.517037 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9d8ee-b1be-4435-9fc4-3c0cc126a7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:27.517321 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9d8ee-b1be-4435-9fc4-3c0cc126a7f9 admin admin] PATH_INFO: `/v3/projects/7c99e53f92aa4ceb91b74dfeac159107` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:27.518708 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c9d8ee-b1be-4435-9fc4-3c0cc126a7f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c99e53f92aa4ceb91b74dfeac159107)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:27.519456 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a9503532-d08e-4a45-801e-3c2d006313ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:27.521424 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c9d8ee-b1be-4435-9fc4-3c0cc126a7f9 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:27.556326 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:27.611139 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:27.653585 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-32c9d8ee-b1be-4435-9fc4-3c0cc126a7f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c99e53f92aa4ceb91b74dfeac159107', 'request_id': 'req-32c9d8ee-b1be-4435-9fc4-3c0cc126a7f9'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:27.671020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-32c9d8ee-b1be-4435-9fc4-3c0cc126a7f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c99e53f92aa4ceb91b74dfeac159107', 'request_id': 'req-32c9d8ee-b1be-4435-9fc4-3c0cc126a7f9'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:27.690650 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-32c9d8ee-b1be-4435-9fc4-3c0cc126a7f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c99e53f92aa4ceb91b74dfeac159107', 'request_id': 'req-32c9d8ee-b1be-4435-9fc4-3c0cc126a7f9'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:27.710174 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-32c9d8ee-b1be-4435-9fc4-3c0cc126a7f9 admin admin] The token cache is being invalidate because project 7c99e53f92aa4ceb91b74dfeac159107 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:27:27.710624 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-32c9d8ee-b1be-4435-9fc4-3c0cc126a7f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:27.711782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-32c9d8ee-b1be-4435-9fc4-3c0cc126a7f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:27.717687 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 846/1687] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:27:27 2022] DELETE /identity/v3/projects/7c99e53f92aa4ceb91b74dfeac159107 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:27.719229 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180aafa7-0de2-41ac-92c3-f05284f492b5 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:27.719513 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180aafa7-0de2-41ac-92c3-f05284f492b5 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:27.719785 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180aafa7-0de2-41ac-92c3-f05284f492b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:27.764951 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 842/1688] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:27 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:27:27.778923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-180aafa7-0de2-41ac-92c3-f05284f492b5 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:27.780233 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-180aafa7-0de2-41ac-92c3-f05284f492b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:27.967412 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 847/1689] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:27.968625 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c719ea9f-8945-4997-aee9-0f8e4fd08e04 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:27.970059 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c719ea9f-8945-4997-aee9-0f8e4fd08e04 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:27.970687 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c719ea9f-8945-4997-aee9-0f8e4fd08e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:28.101597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-c719ea9f-8945-4997-aee9-0f8e4fd08e04 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:28.107047 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-c719ea9f-8945-4997-aee9-0f8e4fd08e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:28.130609 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 843/1690] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:27:28.132278 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180aafa7-0de2-41ac-92c3-f05284f492b5 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:28.149913 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180aafa7-0de2-41ac-92c3-f05284f492b5 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:28.172648 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a437557-3a94-4193-bcb2-3117aecc9d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:28.173533 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a437557-3a94-4193-bcb2-3117aecc9d45 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:28.173673 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a437557-3a94-4193-bcb2-3117aecc9d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:28.174681 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a437557-3a94-4193-bcb2-3117aecc9d45 admin admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:28.174681 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a437557-3a94-4193-bcb2-3117aecc9d45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:27:28.174862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a437557-3a94-4193-bcb2-3117aecc9d45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:28.179559 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a437557-3a94-4193-bcb2-3117aecc9d45 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:28.218600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c719ea9f-8945-4997-aee9-0f8e4fd08e04 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:28.219504 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 848/1691] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:27:28.231334 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c719ea9f-8945-4997-aee9-0f8e4fd08e04 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:28.257834 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de2295-e46f-47c7-a5ba-474525ae453a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:28.258464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de2295-e46f-47c7-a5ba-474525ae453a admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:28.258733 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de2295-e46f-47c7-a5ba-474525ae453a admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:28.259008 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de2295-e46f-47c7-a5ba-474525ae453a admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:28.259635 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de2295-e46f-47c7-a5ba-474525ae453a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:28.261645 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de2295-e46f-47c7-a5ba-474525ae453a admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:28.300917 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a437557-3a94-4193-bcb2-3117aecc9d45 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:28.304239 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-8a437557-3a94-4193-bcb2-3117aecc9d45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:28.311616 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 844/1692] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:27:28 2022] POST /identity/v3/projects => generated 379 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:28.313741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de2295-e46f-47c7-a5ba-474525ae453a admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:28.326107 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a437557-3a94-4193-bcb2-3117aecc9d45 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:28.429452 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de2295-e46f-47c7-a5ba-474525ae453a admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:28.438830 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6bd7a3-cc2f-4cc5-bc4c-57e831878594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:28.440271 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bd7a3-cc2f-4cc5-bc4c-57e831878594 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:28.440776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bd7a3-cc2f-4cc5-bc4c-57e831878594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:28.441251 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bd7a3-cc2f-4cc5-bc4c-57e831878594 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:28.442117 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6bd7a3-cc2f-4cc5-bc4c-57e831878594 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:28.444918 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6bd7a3-cc2f-4cc5-bc4c-57e831878594 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:28.476193 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e73d785-66b6-46cf-8a42-639ba0f6dc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:28.478265 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73d785-66b6-46cf-8a42-639ba0f6dc8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:28.479048 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73d785-66b6-46cf-8a42-639ba0f6dc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:28.479494 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73d785-66b6-46cf-8a42-639ba0f6dc8f admin admin] PATH_INFO: `/v3/users/607f65000c0f46a48a6c6b5e16b69c46` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:28.485166 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e73d785-66b6-46cf-8a42-639ba0f6dc8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=607f65000c0f46a48a6c6b5e16b69c46)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:28.488981 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e73d785-66b6-46cf-8a42-639ba0f6dc8f admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:28.530815 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 845/1693] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:27:28 2022] POST /identity/v3/users => generated 290 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:28.534651 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6bd7a3-cc2f-4cc5-bc4c-57e831878594 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:28.552364 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6bd7a3-cc2f-4cc5-bc4c-57e831878594 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:28.566394 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d3584-2729-44c0-a6a1-0448eec2ffda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:28.567944 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d3584-2729-44c0-a6a1-0448eec2ffda admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:28.568675 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d3584-2729-44c0-a6a1-0448eec2ffda admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:28.572717 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d3584-2729-44c0-a6a1-0448eec2ffda admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:28.575933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d3584-2729-44c0-a6a1-0448eec2ffda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:28.583815 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d3584-2729-44c0-a6a1-0448eec2ffda admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:28.598852 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:28.605376 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-3e73d785-66b6-46cf-8a42-639ba0f6dc8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '607f65000c0f46a48a6c6b5e16b69c46'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:28.614255 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 846/1694] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:28.625780 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-3e73d785-66b6-46cf-8a42-639ba0f6dc8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '607f65000c0f46a48a6c6b5e16b69c46'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:28.626798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d3584-2729-44c0-a6a1-0448eec2ffda admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:28.651141 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d3584-2729-44c0-a6a1-0448eec2ffda admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:28.667998 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-3e73d785-66b6-46cf-8a42-639ba0f6dc8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '607f65000c0f46a48a6c6b5e16b69c46'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:28.668981 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-3e73d785-66b6-46cf-8a42-639ba0f6dc8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '607f65000c0f46a48a6c6b5e16b69c46'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:28.669294 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5b8d2-4e35-4a3b-bebd-ffbacd410336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:28.670053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5b8d2-4e35-4a3b-bebd-ffbacd410336 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:28.670522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5b8d2-4e35-4a3b-bebd-ffbacd410336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:28.670970 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5b8d2-4e35-4a3b-bebd-ffbacd410336 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:28.671720 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5b8d2-4e35-4a3b-bebd-ffbacd410336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:28.674390 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5b8d2-4e35-4a3b-bebd-ffbacd410336 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:28.693472 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-3e73d785-66b6-46cf-8a42-639ba0f6dc8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '607f65000c0f46a48a6c6b5e16b69c46'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:28.698455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 847/1695] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:28.699595 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5b8d2-4e35-4a3b-bebd-ffbacd410336 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:28.707035 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 849/1696] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:27:28 2022] DELETE /identity/v3/users/607f65000c0f46a48a6c6b5e16b69c46 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:28.708595 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-20b5b8d2-4e35-4a3b-bebd-ffbacd410336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:28.721138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908c52f-93bc-4160-ae00-6301c0892391 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:28.723081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908c52f-93bc-4160-ae00-6301c0892391 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:28.724340 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908c52f-93bc-4160-ae00-6301c0892391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:28.732171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5b8d2-4e35-4a3b-bebd-ffbacd410336 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:28.772609 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-d908c52f-93bc-4160-ae00-6301c0892391 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:28.776414 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-d908c52f-93bc-4160-ae00-6301c0892391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:28.852631 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:28.889117 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1f6cf-50d1-4d1e-a9e8-42a1e75915b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:28.889810 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1f6cf-50d1-4d1e-a9e8-42a1e75915b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:28.890320 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1f6cf-50d1-4d1e-a9e8-42a1e75915b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:28.890542 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1f6cf-50d1-4d1e-a9e8-42a1e75915b6 admin admin] PATH_INFO: `/v3/projects/197827793ebf446e95158012bfbfc0d9/users/76e14e6df487455f86e69d2939676bbb/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:28.934951 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec1f6cf-50d1-4d1e-a9e8-42a1e75915b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=197827793ebf446e95158012bfbfc0d9, user_id=76e14e6df487455f86e69d2939676bbb, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:28.940846 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec1f6cf-50d1-4d1e-a9e8-42a1e75915b6 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:28.946165 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 850/1697] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:28.964691 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 848/1698] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:28 2022] PUT /identity/v3/projects/197827793ebf446e95158012bfbfc0d9/users/76e14e6df487455f86e69d2939676bbb/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:28.978659 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b0278-afbc-4b8c-9e31-0d928e347891 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:28.980099 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b0278-afbc-4b8c-9e31-0d928e347891 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:28.980395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b0278-afbc-4b8c-9e31-0d928e347891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:28.981594 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1f6cf-50d1-4d1e-a9e8-42a1e75915b6 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:28.984089 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-1ec1f6cf-50d1-4d1e-a9e8-42a1e75915b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:29.052503 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-177b0278-afbc-4b8c-9e31-0d928e347891 None None] MFA Rules not processed for user `76e14e6df487455f86e69d2939676bbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:29.056507 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-177b0278-afbc-4b8c-9e31-0d928e347891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:29.130595 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1f6cf-50d1-4d1e-a9e8-42a1e75915b6 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:29.153168 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261221d-cbfb-4882-8888-8690ba266c29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:29.153863 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261221d-cbfb-4882-8888-8690ba266c29 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:29.154915 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261221d-cbfb-4882-8888-8690ba266c29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:29.155195 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261221d-cbfb-4882-8888-8690ba266c29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:29.252688 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 851/1699] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:29.253653 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b0278-afbc-4b8c-9e31-0d928e347891 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:29.321201 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3261221d-cbfb-4882-8888-8690ba266c29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:29.322974 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3261221d-cbfb-4882-8888-8690ba266c29 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:29.352740 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 849/1700] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:29.378421 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:29.380820 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b0278-afbc-4b8c-9e31-0d928e347891 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:29.388576 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e7bfd-16a9-4e7b-9fe8-9086d032a413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:29.389426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e7bfd-16a9-4e7b-9fe8-9086d032a413 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:29.392796 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e7bfd-16a9-4e7b-9fe8-9086d032a413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:29.393296 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e7bfd-16a9-4e7b-9fe8-9086d032a413 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:29.394454 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e7bfd-16a9-4e7b-9fe8-9086d032a413 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:29.397075 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e7bfd-16a9-4e7b-9fe8-9086d032a413 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:29.451512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261221d-cbfb-4882-8888-8690ba266c29 service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:29.458875 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 852/1701] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:27:29 2022] POST /identity/v3/projects => generated 381 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:29.467737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261221d-cbfb-4882-8888-8690ba266c29 service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:29.582956 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81b1a2-778e-4433-bda9-15a7beb17e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:29.583788 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81b1a2-778e-4433-bda9-15a7beb17e34 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:29.584270 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81b1a2-778e-4433-bda9-15a7beb17e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:29.584691 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81b1a2-778e-4433-bda9-15a7beb17e34 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:29.585613 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81b1a2-778e-4433-bda9-15a7beb17e34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:29.612996 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81b1a2-778e-4433-bda9-15a7beb17e34 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:29.673504 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 850/1702] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:27:29 2022] POST /identity/v3/users => generated 297 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:29.677078 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e7bfd-16a9-4e7b-9fe8-9086d032a413 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:29.693752 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e7bfd-16a9-4e7b-9fe8-9086d032a413 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:29.708930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a942fa-5eb8-4d7b-83f9-f690851ca440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:29.713086 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a942fa-5eb8-4d7b-83f9-f690851ca440 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:29.713721 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a942fa-5eb8-4d7b-83f9-f690851ca440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:29.714143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a942fa-5eb8-4d7b-83f9-f690851ca440 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:29.715445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a942fa-5eb8-4d7b-83f9-f690851ca440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:29.735620 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a942fa-5eb8-4d7b-83f9-f690851ca440 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:29.760943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81b1a2-778e-4433-bda9-15a7beb17e34 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:29.771397 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 853/1703] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:29.774354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81b1a2-778e-4433-bda9-15a7beb17e34 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:29.782555 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc24e8ad-c326-4506-8e5d-30bbff1eee52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:29.783265 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24e8ad-c326-4506-8e5d-30bbff1eee52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:29.783616 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24e8ad-c326-4506-8e5d-30bbff1eee52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:29.783895 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24e8ad-c326-4506-8e5d-30bbff1eee52 admin admin] PATH_INFO: `/v3/projects/7b5c201ac7424cc2a9aeea2268c30f47/users/3aeaeacb4f964c41a21e7fa7b0e31f7f/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:29.794619 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:29.823995 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc24e8ad-c326-4506-8e5d-30bbff1eee52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b5c201ac7424cc2a9aeea2268c30f47, user_id=3aeaeacb4f964c41a21e7fa7b0e31f7f, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:29.831893 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc24e8ad-c326-4506-8e5d-30bbff1eee52 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:29.862827 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 851/1704] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:29 2022] PUT /identity/v3/projects/7b5c201ac7424cc2a9aeea2268c30f47/users/3aeaeacb4f964c41a21e7fa7b0e31f7f/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:29.868552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a942fa-5eb8-4d7b-83f9-f690851ca440 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:29.887042 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a942fa-5eb8-4d7b-83f9-f690851ca440 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:30.021606 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fdc10f-8a91-4afc-b76d-215ffb2a5ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:30.022814 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fdc10f-8a91-4afc-b76d-215ffb2a5ded admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:30.023351 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fdc10f-8a91-4afc-b76d-215ffb2a5ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:30.023775 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fdc10f-8a91-4afc-b76d-215ffb2a5ded admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:30.026329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fdc10f-8a91-4afc-b76d-215ffb2a5ded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:30.032501 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fdc10f-8a91-4afc-b76d-215ffb2a5ded admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:30.051492 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 854/1705] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:30.055865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc24e8ad-c326-4506-8e5d-30bbff1eee52 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:30.075081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc24e8ad-c326-4506-8e5d-30bbff1eee52 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:30.082246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38d7ab-a102-48e8-9d4c-803c43969ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:30.083179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38d7ab-a102-48e8-9d4c-803c43969ba7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:30.083457 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38d7ab-a102-48e8-9d4c-803c43969ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:30.083729 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38d7ab-a102-48e8-9d4c-803c43969ba7 admin admin] PATH_INFO: `/v3/projects/7b5c201ac7424cc2a9aeea2268c30f47/users/3aeaeacb4f964c41a21e7fa7b0e31f7f/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:30.087834 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38d7ab-a102-48e8-9d4c-803c43969ba7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b5c201ac7424cc2a9aeea2268c30f47, user_id=3aeaeacb4f964c41a21e7fa7b0e31f7f, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:30.090685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38d7ab-a102-48e8-9d4c-803c43969ba7 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:30.115090 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 852/1706] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:30 2022] PUT /identity/v3/projects/7b5c201ac7424cc2a9aeea2268c30f47/users/3aeaeacb4f964c41a21e7fa7b0e31f7f/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:27:30.117277 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9baada-c881-4301-89c9-d93d4201d4d0 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:30.117569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9baada-c881-4301-89c9-d93d4201d4d0 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:30.117846 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9baada-c881-4301-89c9-d93d4201d4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:30.160326 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-ca9baada-c881-4301-89c9-d93d4201d4d0 None None] MFA Rules not processed for user `3aeaeacb4f964c41a21e7fa7b0e31f7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:30.161890 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-ca9baada-c881-4301-89c9-d93d4201d4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:30.264207 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 855/1707] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:30 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:30.276492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38d7ab-a102-48e8-9d4c-803c43969ba7 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:30.278266 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-9c38d7ab-a102-48e8-9d4c-803c43969ba7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:30.369046 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38d7ab-a102-48e8-9d4c-803c43969ba7 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:30.374323 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b259b-82d3-4d75-874b-591d4741a20b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:30.375739 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b259b-82d3-4d75-874b-591d4741a20b service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:30.376440 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b259b-82d3-4d75-874b-591d4741a20b service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:30.380682 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b259b-82d3-4d75-874b-591d4741a20b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:30.395778 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b259b-82d3-4d75-874b-591d4741a20b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:30.411516 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b259b-82d3-4d75-874b-591d4741a20b service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:30.439460 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 853/1708] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:30 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:30.549953 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e96ef7-bb3c-4dd2-a3a0-5ef8ceec7073 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:30.550721 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e96ef7-bb3c-4dd2-a3a0-5ef8ceec7073 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:30.550721 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e96ef7-bb3c-4dd2-a3a0-5ef8ceec7073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:30.601973 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-86e96ef7-bb3c-4dd2-a3a0-5ef8ceec7073 None None] MFA Rules not processed for user `658ce95360454ea1a96a22af310e5a4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:30.603172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-86e96ef7-bb3c-4dd2-a3a0-5ef8ceec7073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:30.607774 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b259b-82d3-4d75-874b-591d4741a20b service cinder] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:30.619280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b259b-82d3-4d75-874b-591d4741a20b service cinder] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:30.629346 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986abca-91b6-4b20-bdb9-c670946fc984 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:30.629976 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986abca-91b6-4b20-bdb9-c670946fc984 service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:30.630248 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986abca-91b6-4b20-bdb9-c670946fc984 service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:30.630517 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986abca-91b6-4b20-bdb9-c670946fc984 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:30.702619 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:30.745005 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 856/1709] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:27:30.764769 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e96ef7-bb3c-4dd2-a3a0-5ef8ceec7073 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:30.771735 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-86e96ef7-bb3c-4dd2-a3a0-5ef8ceec7073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:30.833246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986abca-91b6-4b20-bdb9-c670946fc984 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:30.838831 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986abca-91b6-4b20-bdb9-c670946fc984 service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:30.887979 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 854/1710] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:30 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:30.977425 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0250ce4-4433-4c2e-87eb-d1b1e8e6060f None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:30.977913 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0250ce4-4433-4c2e-87eb-d1b1e8e6060f None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:30.978259 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0250ce4-4433-4c2e-87eb-d1b1e8e6060f None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:30.980065 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 855/1711] 10.222.0.18 () {54 vars in 921 bytes} [Tue Aug 23 04:27:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:27:30.991996 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e96ef7-bb3c-4dd2-a3a0-5ef8ceec7073 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:30.998161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa90e58-b99c-4619-870d-d96abdab3586 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:30.998923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa90e58-b99c-4619-870d-d96abdab3586 service nova] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:30.999269 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa90e58-b99c-4619-870d-d96abdab3586 service nova] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:30.999611 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa90e58-b99c-4619-870d-d96abdab3586 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:31.008544 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa90e58-b99c-4619-870d-d96abdab3586 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:31.012012 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa90e58-b99c-4619-870d-d96abdab3586 service nova] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:31.014011 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81393274-c8c5-4575-98bc-ece6536a645b None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:31.014368 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81393274-c8c5-4575-98bc-ece6536a645b None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:31.014706 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81393274-c8c5-4575-98bc-ece6536a645b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:31.045920 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 857/1712] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:31.047567 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa90e58-b99c-4619-870d-d96abdab3586 service nova] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:31.056840 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-eaa90e58-b99c-4619-870d-d96abdab3586 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:31.066718 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-81393274-c8c5-4575-98bc-ece6536a645b None None] MFA Rules not processed for user `dd503fe3cf6d428e962c14ab864ee608`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:31.070634 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-81393274-c8c5-4575-98bc-ece6536a645b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:31.234228 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 856/1713] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Aug 23 04:27:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:31.285576 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e6f12-f1af-4661-a387-559173f3c906 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:31.285954 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa90e58-b99c-4619-870d-d96abdab3586 service nova] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:31.286075 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e6f12-f1af-4661-a387-559173f3c906 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:31.286427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e6f12-f1af-4661-a387-559173f3c906 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:31.289878 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 857/1714] 10.222.0.18 () {54 vars in 961 bytes} [Tue Aug 23 04:27:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:27:31.295559 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336987c8-4bd4-4938-8a1b-5a9a38796dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:31.296249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336987c8-4bd4-4938-8a1b-5a9a38796dfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:31.296531 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336987c8-4bd4-4938-8a1b-5a9a38796dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:31.296807 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336987c8-4bd4-4938-8a1b-5a9a38796dfe admin admin] PATH_INFO: `/v3/projects/a85a378e0ec04d898c8b9f23eb097457` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:31.298513 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336987c8-4bd4-4938-8a1b-5a9a38796dfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=a85a378e0ec04d898c8b9f23eb097457)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:31.300565 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336987c8-4bd4-4938-8a1b-5a9a38796dfe admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:31.307491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed484244-2b52-402a-a65b-f0c9bfdd266a None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:31.307846 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed484244-2b52-402a-a65b-f0c9bfdd266a None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:31.308233 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed484244-2b52-402a-a65b-f0c9bfdd266a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:31.370404 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-ed484244-2b52-402a-a65b-f0c9bfdd266a None None] MFA Rules not processed for user `dd503fe3cf6d428e962c14ab864ee608`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:31.372782 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-ed484244-2b52-402a-a65b-f0c9bfdd266a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:31.411560 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 858/1715] 10.222.0.18 () {58 vars in 1058 bytes} [Tue Aug 23 04:27:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:27:31.414193 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed484244-2b52-402a-a65b-f0c9bfdd266a None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:31.417211 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-336987c8-4bd4-4938-8a1b-5a9a38796dfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a85a378e0ec04d898c8b9f23eb097457', 'request_id': 'req-336987c8-4bd4-4938-8a1b-5a9a38796dfe'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:31.427483 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:31.437093 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-336987c8-4bd4-4938-8a1b-5a9a38796dfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a85a378e0ec04d898c8b9f23eb097457', 'request_id': 'req-336987c8-4bd4-4938-8a1b-5a9a38796dfe'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:31.438528 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed484244-2b52-402a-a65b-f0c9bfdd266a None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:31.451405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-336987c8-4bd4-4938-8a1b-5a9a38796dfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a85a378e0ec04d898c8b9f23eb097457', 'request_id': 'req-336987c8-4bd4-4938-8a1b-5a9a38796dfe'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:31.457501 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:31.476132 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-336987c8-4bd4-4938-8a1b-5a9a38796dfe admin admin] The token cache is being invalidate because project a85a378e0ec04d898c8b9f23eb097457 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:27:31.476575 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-336987c8-4bd4-4938-8a1b-5a9a38796dfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:31.482635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-336987c8-4bd4-4938-8a1b-5a9a38796dfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:31.484626 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 858/1716] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:27:31 2022] DELETE /identity/v3/projects/a85a378e0ec04d898c8b9f23eb097457 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:31.488819 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336987c8-4bd4-4938-8a1b-5a9a38796dfe admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:31.490617 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-336987c8-4bd4-4938-8a1b-5a9a38796dfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:31.654449 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aabc49-9962-471e-9aff-aa1bf944dc61 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd503fe3cf6d428e962c14ab864ee608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:31.666545 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aabc49-9962-471e-9aff-aa1bf944dc61 service placement] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:31.667524 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aabc49-9962-471e-9aff-aa1bf944dc61 service placement] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:31.669863 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aabc49-9962-471e-9aff-aa1bf944dc61 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:31.676213 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-73aabc49-9962-471e-9aff-aa1bf944dc61 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:31.694594 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336987c8-4bd4-4938-8a1b-5a9a38796dfe admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:31.708181 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303a4f2-e68b-4b8c-8fa4-fb2003bd0dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:31.710700 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303a4f2-e68b-4b8c-8fa4-fb2003bd0dfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:31.710833 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303a4f2-e68b-4b8c-8fa4-fb2003bd0dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:31.711526 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303a4f2-e68b-4b8c-8fa4-fb2003bd0dfc admin admin] PATH_INFO: `/v3/users/b1505570d28447fab698ed154e55e623` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:31.715008 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aabc49-9962-471e-9aff-aa1bf944dc61 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:31.721285 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aabc49-9962-471e-9aff-aa1bf944dc61 service placement] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:31.722875 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5303a4f2-e68b-4b8c-8fa4-fb2003bd0dfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1505570d28447fab698ed154e55e623)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:31.727480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5303a4f2-e68b-4b8c-8fa4-fb2003bd0dfc admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:31.787330 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 859/1717] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:31.878085 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-5303a4f2-e68b-4b8c-8fa4-fb2003bd0dfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1505570d28447fab698ed154e55e623'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:31.907701 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-5303a4f2-e68b-4b8c-8fa4-fb2003bd0dfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1505570d28447fab698ed154e55e623'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:31.943788 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-5303a4f2-e68b-4b8c-8fa4-fb2003bd0dfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1505570d28447fab698ed154e55e623'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:31.944646 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-5303a4f2-e68b-4b8c-8fa4-fb2003bd0dfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1505570d28447fab698ed154e55e623'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:31.952899 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-5303a4f2-e68b-4b8c-8fa4-fb2003bd0dfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1505570d28447fab698ed154e55e623'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:31.968189 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 859/1718] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:27:31 2022] DELETE /identity/v3/users/b1505570d28447fab698ed154e55e623 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:27:32.048636 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0216bc9-5151-4ff4-ab5c-7413eff01ad1 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:32.048966 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0216bc9-5151-4ff4-ab5c-7413eff01ad1 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:32.049190 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0216bc9-5151-4ff4-ab5c-7413eff01ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:32.116172 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-e0216bc9-5151-4ff4-ab5c-7413eff01ad1 None None] MFA Rules not processed for user `bafdb895190e45ba96e71fd348c41c3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:32.120937 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-e0216bc9-5151-4ff4-ab5c-7413eff01ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:32.260432 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 860/1719] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:32.272055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c9182-7f5c-4b15-b8d8-03f7f95d1c8b None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:32.272739 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c9182-7f5c-4b15-b8d8-03f7f95d1c8b None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:32.273089 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c9182-7f5c-4b15-b8d8-03f7f95d1c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:32.336044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-e38c9182-7f5c-4b15-b8d8-03f7f95d1c8b None None] MFA Rules not processed for user `bafdb895190e45ba96e71fd348c41c3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:32.337477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-e38c9182-7f5c-4b15-b8d8-03f7f95d1c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:32.361048 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 860/1720] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:32.374616 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0216bc9-5151-4ff4-ab5c-7413eff01ad1 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:32.385504 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-e0216bc9-5151-4ff4-ab5c-7413eff01ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:32.389434 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72df18d-18ff-4443-b886-59c0e926766c None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:32.389762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72df18d-18ff-4443-b886-59c0e926766c None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:32.390030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72df18d-18ff-4443-b886-59c0e926766c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:32.488002 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-b72df18d-18ff-4443-b886-59c0e926766c None None] MFA Rules not processed for user `820b9a99be574eb8b09629988c4c59c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:32.489511 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-b72df18d-18ff-4443-b886-59c0e926766c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:32.614757 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0216bc9-5151-4ff4-ab5c-7413eff01ad1 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:32.631497 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5f25c-9bae-4bb4-9d32-0de8e16af4a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:32.635320 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5f25c-9bae-4bb4-9d32-0de8e16af4a3 service nova] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:32.637438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5f25c-9bae-4bb4-9d32-0de8e16af4a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:32.638650 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5f25c-9bae-4bb4-9d32-0de8e16af4a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:32.665299 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:32.673199 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d5f25c-9bae-4bb4-9d32-0de8e16af4a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:32.674579 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d5f25c-9bae-4bb4-9d32-0de8e16af4a3 service nova] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:32.694550 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 861/1721] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:32.705598 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024dc2b-f266-4485-a28a-b68d3c91a241 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:32.706680 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024dc2b-f266-4485-a28a-b68d3c91a241 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:32.707176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024dc2b-f266-4485-a28a-b68d3c91a241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:32.728353 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 861/1722] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:32.766167 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-c024dc2b-f266-4485-a28a-b68d3c91a241 None None] MFA Rules not processed for user `820b9a99be574eb8b09629988c4c59c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:32.768239 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-c024dc2b-f266-4485-a28a-b68d3c91a241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:32.796160 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 862/1723] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:32.800486 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5f25c-9bae-4bb4-9d32-0de8e16af4a3 service nova] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:32.802093 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-37d5f25c-9bae-4bb4-9d32-0de8e16af4a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:32.841446 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5f25c-9bae-4bb4-9d32-0de8e16af4a3 service nova] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:32.863513 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f3c3a2-b50b-4691-bde0-77c796a1a527 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:32.864518 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3c3a2-b50b-4691-bde0-77c796a1a527 service nova] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:32.865535 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3c3a2-b50b-4691-bde0-77c796a1a527 service nova] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:32.865535 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3c3a2-b50b-4691-bde0-77c796a1a527 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:32.880656 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:32.889593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f3c3a2-b50b-4691-bde0-77c796a1a527 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:32.891222 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f3c3a2-b50b-4691-bde0-77c796a1a527 service nova] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:32.922937 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 862/1724] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:32 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:33.863355 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c024dc2b-f266-4485-a28a-b68d3c91a241 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:33.865726 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-c024dc2b-f266-4485-a28a-b68d3c91a241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:34.073211 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c024dc2b-f266-4485-a28a-b68d3c91a241 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:34.093202 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316892f-f57f-4658-be79-8d729982f5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:34.094193 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316892f-f57f-4658-be79-8d729982f5c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:34.094652 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316892f-f57f-4658-be79-8d729982f5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:34.095027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316892f-f57f-4658-be79-8d729982f5c8 admin admin] PATH_INFO: `/v3/projects/62d5fe6813d34d36a11e3b7b6239b2e4` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:34.097181 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d316892f-f57f-4658-be79-8d729982f5c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62d5fe6813d34d36a11e3b7b6239b2e4)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:34.099588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d316892f-f57f-4658-be79-8d729982f5c8 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:34.180455 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d316892f-f57f-4658-be79-8d729982f5c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62d5fe6813d34d36a11e3b7b6239b2e4', 'request_id': 'req-d316892f-f57f-4658-be79-8d729982f5c8'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:34.188579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d316892f-f57f-4658-be79-8d729982f5c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62d5fe6813d34d36a11e3b7b6239b2e4', 'request_id': 'req-d316892f-f57f-4658-be79-8d729982f5c8'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:34.223866 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d316892f-f57f-4658-be79-8d729982f5c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62d5fe6813d34d36a11e3b7b6239b2e4', 'request_id': 'req-d316892f-f57f-4658-be79-8d729982f5c8'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:34.255159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d316892f-f57f-4658-be79-8d729982f5c8 admin admin] The token cache is being invalidate because project 62d5fe6813d34d36a11e3b7b6239b2e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:27:34.256065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d316892f-f57f-4658-be79-8d729982f5c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:34.257028 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d316892f-f57f-4658-be79-8d729982f5c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:34.259196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 863/1725] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:27:33 2022] DELETE /identity/v3/projects/62d5fe6813d34d36a11e3b7b6239b2e4 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:34.264658 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cbaf37-3ded-4768-b448-7a0b13d82fa7 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:34.265150 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cbaf37-3ded-4768-b448-7a0b13d82fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:34.265229 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cbaf37-3ded-4768-b448-7a0b13d82fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:34.317807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-b3cbaf37-3ded-4768-b448-7a0b13d82fa7 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:34.319057 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-b3cbaf37-3ded-4768-b448-7a0b13d82fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:34.370399 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a48c27-72ec-48b3-80f3-b08d7156586d None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:34.370764 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a48c27-72ec-48b3-80f3-b08d7156586d None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:34.371132 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a48c27-72ec-48b3-80f3-b08d7156586d None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:34.373634 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 864/1726] 10.222.0.18 () {54 vars in 912 bytes} [Tue Aug 23 04:27:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:27:34.380223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f4926-d7ec-4aa3-8eb6-e464ff87372b None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:34.380659 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f4926-d7ec-4aa3-8eb6-e464ff87372b None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:34.381004 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f4926-d7ec-4aa3-8eb6-e464ff87372b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:34.541503 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 863/1727] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:34.541503 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347d5ed-c86a-4382-9155-7de8e5ca26b9 None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:34.541503 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347d5ed-c86a-4382-9155-7de8e5ca26b9 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:34.541503 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347d5ed-c86a-4382-9155-7de8e5ca26b9 None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:34.544938 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 864/1728] 10.222.0.18 () {54 vars in 912 bytes} [Tue Aug 23 04:27:34 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:27:34.546317 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80a555-6bfb-40fb-b77a-29630fa6dbe7 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:34.546600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80a555-6bfb-40fb-b77a-29630fa6dbe7 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:34.547238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80a555-6bfb-40fb-b77a-29630fa6dbe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:34.565092 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-749f4926-d7ec-4aa3-8eb6-e464ff87372b None None] MFA Rules not processed for user `fdca9c48e5d740f6bda8c48072b54c0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:34.566427 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-749f4926-d7ec-4aa3-8eb6-e464ff87372b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:34.577783 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-5a80a555-6bfb-40fb-b77a-29630fa6dbe7 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:34.580005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-5a80a555-6bfb-40fb-b77a-29630fa6dbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:34.595364 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 865/1729] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:34.597683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea854f-860a-4c35-a311-1367c7526ded None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:34.598910 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea854f-860a-4c35-a311-1367c7526ded None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:34.599416 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea854f-860a-4c35-a311-1367c7526ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:34.633601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-8cea854f-860a-4c35-a311-1367c7526ded None None] MFA Rules not processed for user `fdca9c48e5d740f6bda8c48072b54c0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:34.639521 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-8cea854f-860a-4c35-a311-1367c7526ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:34.750279 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 865/1730] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Aug 23 04:27:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:34.752637 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f4926-d7ec-4aa3-8eb6-e464ff87372b None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:34.782864 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f4926-d7ec-4aa3-8eb6-e464ff87372b None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:34.797567 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3b0d6-2a1f-4843-a7cd-04400b771ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:34.798207 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3b0d6-2a1f-4843-a7cd-04400b771ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:34.798627 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3b0d6-2a1f-4843-a7cd-04400b771ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:34.798998 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3b0d6-2a1f-4843-a7cd-04400b771ce1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:34.800055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a3b0d6-2a1f-4843-a7cd-04400b771ce1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:27:34.801193 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a3b0d6-2a1f-4843-a7cd-04400b771ce1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:34.806987 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a3b0d6-2a1f-4843-a7cd-04400b771ce1 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:34.822131 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 866/1731] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Aug 23 04:27:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:34.832219 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78852900-5fe9-4aa1-b634-20e5bbcf3fb3 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:34.832445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:34.832659 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78852900-5fe9-4aa1-b634-20e5bbcf3fb3 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:34.832845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78852900-5fe9-4aa1-b634-20e5bbcf3fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:34.850975 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 866/1732] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:27:34.853541 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3b0d6-2a1f-4843-a7cd-04400b771ce1 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:34.855558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-51a3b0d6-2a1f-4843-a7cd-04400b771ce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:34.898156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3b0d6-2a1f-4843-a7cd-04400b771ce1 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:34.913724 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce29b87-d934-4ecb-a39e-138be5cb0889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:34.915066 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce29b87-d934-4ecb-a39e-138be5cb0889 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:34.915404 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce29b87-d934-4ecb-a39e-138be5cb0889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:34.915610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce29b87-d934-4ecb-a39e-138be5cb0889 admin admin] PATH_INFO: `/v3/users/35011187d3f14a68ba26681354c8e32d` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:34.918970 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce29b87-d934-4ecb-a39e-138be5cb0889 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35011187d3f14a68ba26681354c8e32d)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:34.925497 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce29b87-d934-4ecb-a39e-138be5cb0889 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:34.944924 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-78852900-5fe9-4aa1-b634-20e5bbcf3fb3 None None] MFA Rules not processed for user `a91872508ecd49d98bd95651412f85a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:34.946201 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-78852900-5fe9-4aa1-b634-20e5bbcf3fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:35.054401 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-bce29b87-d934-4ecb-a39e-138be5cb0889 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35011187d3f14a68ba26681354c8e32d'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:35.070999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-bce29b87-d934-4ecb-a39e-138be5cb0889 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35011187d3f14a68ba26681354c8e32d'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:35.100390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-bce29b87-d934-4ecb-a39e-138be5cb0889 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35011187d3f14a68ba26681354c8e32d'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:35.108848 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-bce29b87-d934-4ecb-a39e-138be5cb0889 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35011187d3f14a68ba26681354c8e32d'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:35.117805 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 867/1733] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:35.127623 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78852900-5fe9-4aa1-b634-20e5bbcf3fb3 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:35.129172 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-78852900-5fe9-4aa1-b634-20e5bbcf3fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:35.137986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-bce29b87-d934-4ecb-a39e-138be5cb0889 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35011187d3f14a68ba26681354c8e32d'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:35.147961 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 867/1734] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:27:34 2022] DELETE /identity/v3/users/35011187d3f14a68ba26681354c8e32d => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:35.149373 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce29b87-d934-4ecb-a39e-138be5cb0889 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:35.151099 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-bce29b87-d934-4ecb-a39e-138be5cb0889 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:35.160242 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78852900-5fe9-4aa1-b634-20e5bbcf3fb3 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:35.224492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:35.279601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d19291-e837-46f6-a168-89bbd01de6bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:35.280621 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d19291-e837-46f6-a168-89bbd01de6bc service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:35.280621 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d19291-e837-46f6-a168-89bbd01de6bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:35.281284 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d19291-e837-46f6-a168-89bbd01de6bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:35.282967 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce29b87-d934-4ecb-a39e-138be5cb0889 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:35.283091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-92d19291-e837-46f6-a168-89bbd01de6bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:35.291580 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6620245f-ead1-4f8c-87b5-21c8c1a3098a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:35.292887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6620245f-ead1-4f8c-87b5-21c8c1a3098a service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:35.293173 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6620245f-ead1-4f8c-87b5-21c8c1a3098a service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:35.295236 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6620245f-ead1-4f8c-87b5-21c8c1a3098a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:35.298413 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d19291-e837-46f6-a168-89bbd01de6bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:35.300097 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d19291-e837-46f6-a168-89bbd01de6bc service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:35.300335 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-6620245f-ead1-4f8c-87b5-21c8c1a3098a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:35.311774 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 868/1735] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:35.312828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d19291-e837-46f6-a168-89bbd01de6bc service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:35.313215 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6620245f-ead1-4f8c-87b5-21c8c1a3098a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:35.314138 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-92d19291-e837-46f6-a168-89bbd01de6bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:35.315197 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6620245f-ead1-4f8c-87b5-21c8c1a3098a service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:35.324340 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 868/1736] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:35.444638 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d19291-e837-46f6-a168-89bbd01de6bc service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:35.457055 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a75f00-6a4d-4310-9064-18319f982c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:35.463014 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a75f00-6a4d-4310-9064-18319f982c43 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:35.463014 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a75f00-6a4d-4310-9064-18319f982c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:35.463014 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a75f00-6a4d-4310-9064-18319f982c43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:35.463014 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a75f00-6a4d-4310-9064-18319f982c43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:35.464756 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a75f00-6a4d-4310-9064-18319f982c43 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:35.521209 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 869/1737] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:27:35 2022] POST /identity/v3/projects => generated 379 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:35.525547 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6620245f-ead1-4f8c-87b5-21c8c1a3098a service neutron] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:35.538710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6620245f-ead1-4f8c-87b5-21c8c1a3098a service neutron] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:35.649734 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20d81f-1ac0-4459-8bde-5e1420f3f8a2 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:35.650026 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20d81f-1ac0-4459-8bde-5e1420f3f8a2 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:35.650304 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20d81f-1ac0-4459-8bde-5e1420f3f8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:35.729564 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-6d20d81f-1ac0-4459-8bde-5e1420f3f8a2 None None] MFA Rules not processed for user `c8fa0859e5c14c53a5a2473c95efe152`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:35.731676 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-6d20d81f-1ac0-4459-8bde-5e1420f3f8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:35.759329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54cf3dd-02dc-42f3-a650-819f6f370de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:35.761244 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cf3dd-02dc-42f3-a650-819f6f370de1 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:35.761921 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cf3dd-02dc-42f3-a650-819f6f370de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:35.762290 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cf3dd-02dc-42f3-a650-819f6f370de1 admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:35.764032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54cf3dd-02dc-42f3-a650-819f6f370de1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:35.771184 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54cf3dd-02dc-42f3-a650-819f6f370de1 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:35.801058 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 869/1738] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:27:35 2022] POST /identity/v3/users => generated 290 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:35.804684 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54cf3dd-02dc-42f3-a650-819f6f370de1 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:35.824116 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:35.844895 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54cf3dd-02dc-42f3-a650-819f6f370de1 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:35.851735 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8bdf7f-9c07-4f34-a5ec-546fd4e5c17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:35.852992 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8bdf7f-9c07-4f34-a5ec-546fd4e5c17a admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:35.853249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8bdf7f-9c07-4f34-a5ec-546fd4e5c17a admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:35.853621 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8bdf7f-9c07-4f34-a5ec-546fd4e5c17a admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:35.854437 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8bdf7f-9c07-4f34-a5ec-546fd4e5c17a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:35.856435 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8bdf7f-9c07-4f34-a5ec-546fd4e5c17a admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:35.878629 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 870/1740] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:35.878829 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 870/1740] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:35 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:35.879779 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d20d81f-1ac0-4459-8bde-5e1420f3f8a2 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:35.895396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d20d81f-1ac0-4459-8bde-5e1420f3f8a2 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:35.900472 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efeb81a-c5fe-4b08-995d-30ac333d4c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:35.901115 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efeb81a-c5fe-4b08-995d-30ac333d4c51 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:35.901393 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efeb81a-c5fe-4b08-995d-30ac333d4c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:35.901665 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efeb81a-c5fe-4b08-995d-30ac333d4c51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:35.902575 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efeb81a-c5fe-4b08-995d-30ac333d4c51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:35.905861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efeb81a-c5fe-4b08-995d-30ac333d4c51 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:35.927185 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 871/1741] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:35.928531 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8bdf7f-9c07-4f34-a5ec-546fd4e5c17a admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:35.943560 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8bdf7f-9c07-4f34-a5ec-546fd4e5c17a admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:35.956069 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e2482-61c2-4c28-b6b8-e199fab9b7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:35.956940 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e2482-61c2-4c28-b6b8-e199fab9b7da admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:35.957320 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e2482-61c2-4c28-b6b8-e199fab9b7da admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:35.957704 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e2482-61c2-4c28-b6b8-e199fab9b7da admin admin] PATH_INFO: `/v3/projects/8660bc7b611147d2be0cf05f802226bf/users/21ad6e06e6f74753aa94eebfed82d009/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:35.986273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e2482-61c2-4c28-b6b8-e199fab9b7da admin admin] RBAC: Authorizing `identity:create_grant(project_id=8660bc7b611147d2be0cf05f802226bf, user_id=21ad6e06e6f74753aa94eebfed82d009, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:35.991220 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e2482-61c2-4c28-b6b8-e199fab9b7da admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:36.033019 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 871/1742] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:35 2022] PUT /identity/v3/projects/8660bc7b611147d2be0cf05f802226bf/users/21ad6e06e6f74753aa94eebfed82d009/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:36.038249 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d00c0-249b-4398-adbb-5d6ee097f3bb None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:36.038696 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d00c0-249b-4398-adbb-5d6ee097f3bb None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:36.039568 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d00c0-249b-4398-adbb-5d6ee097f3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:36.102377 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-271d00c0-249b-4398-adbb-5d6ee097f3bb None None] MFA Rules not processed for user `21ad6e06e6f74753aa94eebfed82d009`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:36.103624 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-271d00c0-249b-4398-adbb-5d6ee097f3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:36.171084 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:36.202084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d3b15-8867-449c-813b-5942ff0ac9cf None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:36.204397 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d3b15-8867-449c-813b-5942ff0ac9cf None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:36.205888 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d3b15-8867-449c-813b-5942ff0ac9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:36.221111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 872/1743] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:36.229480 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d00c0-249b-4398-adbb-5d6ee097f3bb None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:36.241539 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-583d3b15-8867-449c-813b-5942ff0ac9cf None None] MFA Rules not processed for user `658ce95360454ea1a96a22af310e5a4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:36.242884 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-583d3b15-8867-449c-813b-5942ff0ac9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:36.336085 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 872/1744] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:27:36.337234 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d3b15-8867-449c-813b-5942ff0ac9cf None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:36.340167 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-583d3b15-8867-449c-813b-5942ff0ac9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:36.414070 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d00c0-249b-4398-adbb-5d6ee097f3bb None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:36.420326 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f3e2e4-23fe-4c93-873b-a63bee1aecbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:36.420966 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3e2e4-23fe-4c93-873b-a63bee1aecbf admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:36.422599 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3e2e4-23fe-4c93-873b-a63bee1aecbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:36.422912 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3e2e4-23fe-4c93-873b-a63bee1aecbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:36.423553 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f3e2e4-23fe-4c93-873b-a63bee1aecbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:36.426613 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f3e2e4-23fe-4c93-873b-a63bee1aecbf admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:36.475515 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 873/1745] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:27:36 2022] POST /identity/v3/projects => generated 379 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:36.479903 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f3e2e4-23fe-4c93-873b-a63bee1aecbf admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:36.526240 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d3b15-8867-449c-813b-5942ff0ac9cf None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:36.630297 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71986228-bccc-4eab-8cc0-b3891111c119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:36.631159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71986228-bccc-4eab-8cc0-b3891111c119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:36.631416 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71986228-bccc-4eab-8cc0-b3891111c119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:36.631686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71986228-bccc-4eab-8cc0-b3891111c119 admin admin] PATH_INFO: `/v3/projects/8a770a11a4d14808b16d52629737a0ce` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:36.640845 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71986228-bccc-4eab-8cc0-b3891111c119 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a770a11a4d14808b16d52629737a0ce)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:36.643377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71986228-bccc-4eab-8cc0-b3891111c119 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:36.674487 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:36.722034 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f3e2e4-23fe-4c93-873b-a63bee1aecbf admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:36.723144 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-71986228-bccc-4eab-8cc0-b3891111c119 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a770a11a4d14808b16d52629737a0ce', 'request_id': 'req-71986228-bccc-4eab-8cc0-b3891111c119'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:36.733909 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-71986228-bccc-4eab-8cc0-b3891111c119 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a770a11a4d14808b16d52629737a0ce', 'request_id': 'req-71986228-bccc-4eab-8cc0-b3891111c119'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:36.745112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-71986228-bccc-4eab-8cc0-b3891111c119 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a770a11a4d14808b16d52629737a0ce', 'request_id': 'req-71986228-bccc-4eab-8cc0-b3891111c119'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:36.772373 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-71986228-bccc-4eab-8cc0-b3891111c119 admin admin] The token cache is being invalidate because project 8a770a11a4d14808b16d52629737a0ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:27:36.772671 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-71986228-bccc-4eab-8cc0-b3891111c119 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:36.773394 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-71986228-bccc-4eab-8cc0-b3891111c119 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:36.775706 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 873/1746] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:27:36 2022] DELETE /identity/v3/projects/8a770a11a4d14808b16d52629737a0ce => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:36.776796 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71986228-bccc-4eab-8cc0-b3891111c119 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:36.778440 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-71986228-bccc-4eab-8cc0-b3891111c119 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:36.912552 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43842d4-0144-4901-9a29-4b4fd72ad2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:36.913448 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43842d4-0144-4901-9a29-4b4fd72ad2cc admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:36.913929 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43842d4-0144-4901-9a29-4b4fd72ad2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:36.914406 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43842d4-0144-4901-9a29-4b4fd72ad2cc admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:36.915241 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43842d4-0144-4901-9a29-4b4fd72ad2cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:36.917447 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43842d4-0144-4901-9a29-4b4fd72ad2cc admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:36.963892 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 874/1747] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:27:36 2022] POST /identity/v3/users => generated 296 bytes in 484 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:27:36.964830 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43842d4-0144-4901-9a29-4b4fd72ad2cc admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:36.968489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-d43842d4-0144-4901-9a29-4b4fd72ad2cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:36.981410 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:36.989616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71986228-bccc-4eab-8cc0-b3891111c119 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:36.990476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43842d4-0144-4901-9a29-4b4fd72ad2cc admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:37.003641 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68d22b-97b8-480b-b44f-fec490f3102e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:37.004952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68d22b-97b8-480b-b44f-fec490f3102e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:37.006756 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68d22b-97b8-480b-b44f-fec490f3102e admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:37.008773 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68d22b-97b8-480b-b44f-fec490f3102e admin admin] PATH_INFO: `/v3/users/61739a2a9d474545a93a81d5a0c31aeb` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:37.010189 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34931f-8aee-4bda-92e0-cbf406239f85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:37.010896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34931f-8aee-4bda-92e0-cbf406239f85 service nova] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:37.011242 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34931f-8aee-4bda-92e0-cbf406239f85 service nova] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:37.011581 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34931f-8aee-4bda-92e0-cbf406239f85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:37.012076 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68d22b-97b8-480b-b44f-fec490f3102e admin admin] RBAC: Authorizing `identity:delete_user(user_id=61739a2a9d474545a93a81d5a0c31aeb)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:37.013550 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-3a34931f-8aee-4bda-92e0-cbf406239f85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:37.014816 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68d22b-97b8-480b-b44f-fec490f3102e admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:37.089040 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-2a68d22b-97b8-480b-b44f-fec490f3102e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61739a2a9d474545a93a81d5a0c31aeb'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:37.089759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-2a68d22b-97b8-480b-b44f-fec490f3102e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61739a2a9d474545a93a81d5a0c31aeb'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:37.107440 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-2a68d22b-97b8-480b-b44f-fec490f3102e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61739a2a9d474545a93a81d5a0c31aeb'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:37.129221 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-2a68d22b-97b8-480b-b44f-fec490f3102e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61739a2a9d474545a93a81d5a0c31aeb'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:37.137966 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-2a68d22b-97b8-480b-b44f-fec490f3102e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61739a2a9d474545a93a81d5a0c31aeb'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:37.146079 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 875/1748] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:27:36 2022] DELETE /identity/v3/users/61739a2a9d474545a93a81d5a0c31aeb => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:37.146984 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68d22b-97b8-480b-b44f-fec490f3102e admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:37.148599 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-2a68d22b-97b8-480b-b44f-fec490f3102e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:37.154283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a34931f-8aee-4bda-92e0-cbf406239f85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:37.169353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a34931f-8aee-4bda-92e0-cbf406239f85 service nova] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:37.197081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 874/1749] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:37.210115 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34931f-8aee-4bda-92e0-cbf406239f85 service nova] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:37.214552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-3a34931f-8aee-4bda-92e0-cbf406239f85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:37.237734 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68d22b-97b8-480b-b44f-fec490f3102e admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:37.245087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85edaf5-758f-4db0-a36b-786a68b0ddef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd503fe3cf6d428e962c14ab864ee608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:37.246334 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85edaf5-758f-4db0-a36b-786a68b0ddef service placement] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:37.246620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85edaf5-758f-4db0-a36b-786a68b0ddef service placement] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:37.246899 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85edaf5-758f-4db0-a36b-786a68b0ddef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:37.249716 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a85edaf5-758f-4db0-a36b-786a68b0ddef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:37.283143 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85edaf5-758f-4db0-a36b-786a68b0ddef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:37.288218 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85edaf5-758f-4db0-a36b-786a68b0ddef service placement] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:37.315404 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 876/1750] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:37.316305 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85edaf5-758f-4db0-a36b-786a68b0ddef service placement] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:37.318339 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a85edaf5-758f-4db0-a36b-786a68b0ddef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:37.350531 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:37.423530 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34931f-8aee-4bda-92e0-cbf406239f85 service nova] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:37.446876 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4466902-4e0f-4e75-8a95-c217dbb8d17d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd503fe3cf6d428e962c14ab864ee608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:37.451704 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4466902-4e0f-4e75-8a95-c217dbb8d17d service placement] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:37.453826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4466902-4e0f-4e75-8a95-c217dbb8d17d service placement] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:37.456524 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4466902-4e0f-4e75-8a95-c217dbb8d17d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:37.496406 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4466902-4e0f-4e75-8a95-c217dbb8d17d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:37.498261 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4466902-4e0f-4e75-8a95-c217dbb8d17d service placement] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:37.528088 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 875/1751] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:37.558993 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85edaf5-758f-4db0-a36b-786a68b0ddef service placement] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:37.567260 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c7b84-e4ba-4f5b-8e41-7568c1fc8848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:37.568237 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c7b84-e4ba-4f5b-8e41-7568c1fc8848 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:37.568585 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c7b84-e4ba-4f5b-8e41-7568c1fc8848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:37.568944 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c7b84-e4ba-4f5b-8e41-7568c1fc8848 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:37.569735 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7c7b84-e4ba-4f5b-8e41-7568c1fc8848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:37.573472 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7c7b84-e4ba-4f5b-8e41-7568c1fc8848 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:37.595348 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:37.606139 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 877/1752] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:37.607979 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4466902-4e0f-4e75-8a95-c217dbb8d17d service placement] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:37.620687 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4466902-4e0f-4e75-8a95-c217dbb8d17d service placement] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:37.631255 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b7365-6f92-461b-a70b-6a92c355f36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:37.633155 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b7365-6f92-461b-a70b-6a92c355f36a admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:37.633695 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b7365-6f92-461b-a70b-6a92c355f36a admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:37.634170 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b7365-6f92-461b-a70b-6a92c355f36a admin admin] PATH_INFO: `/v3/projects/963387eedd004e16a27e452a06fa73b4/users/fe85040c0552463ea64e2e90b0a25466/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:37.657142 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30b7365-6f92-461b-a70b-6a92c355f36a admin admin] RBAC: Authorizing `identity:create_grant(project_id=963387eedd004e16a27e452a06fa73b4, user_id=fe85040c0552463ea64e2e90b0a25466, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:37.661339 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30b7365-6f92-461b-a70b-6a92c355f36a admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:37.704183 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 876/1753] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:37 2022] PUT /identity/v3/projects/963387eedd004e16a27e452a06fa73b4/users/fe85040c0552463ea64e2e90b0a25466/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:37.704908 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c7b84-e4ba-4f5b-8e41-7568c1fc8848 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:37.722469 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c7b84-e4ba-4f5b-8e41-7568c1fc8848 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:37.838609 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b7365-6f92-461b-a70b-6a92c355f36a admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:37.840756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-e30b7365-6f92-461b-a70b-6a92c355f36a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:37.904575 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797fc258-98d7-4e20-a661-b1c61a6b9247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:37.905224 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fc258-98d7-4e20-a661-b1c61a6b9247 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:37.906740 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fc258-98d7-4e20-a661-b1c61a6b9247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:37.907050 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fc258-98d7-4e20-a661-b1c61a6b9247 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:37.908984 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797fc258-98d7-4e20-a661-b1c61a6b9247 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:37.910670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797fc258-98d7-4e20-a661-b1c61a6b9247 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:37.936948 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 878/1754] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:27:37.939719 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797fc258-98d7-4e20-a661-b1c61a6b9247 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:37.966772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797fc258-98d7-4e20-a661-b1c61a6b9247 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:37.981770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ee066-767b-4e37-ba5e-a4a4bb18989d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:37.982466 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ee066-767b-4e37-ba5e-a4a4bb18989d admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:37.982744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ee066-767b-4e37-ba5e-a4a4bb18989d admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:37.983020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ee066-767b-4e37-ba5e-a4a4bb18989d admin admin] PATH_INFO: `/v3/projects/963387eedd004e16a27e452a06fa73b4/users/fe85040c0552463ea64e2e90b0a25466/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:37.987634 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ee066-767b-4e37-ba5e-a4a4bb18989d admin admin] RBAC: Authorizing `identity:create_grant(project_id=963387eedd004e16a27e452a06fa73b4, user_id=fe85040c0552463ea64e2e90b0a25466, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:37.990894 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ee066-767b-4e37-ba5e-a4a4bb18989d admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:38.020359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 879/1755] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:37 2022] PUT /identity/v3/projects/963387eedd004e16a27e452a06fa73b4/users/fe85040c0552463ea64e2e90b0a25466/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:38.038608 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b7365-6f92-461b-a70b-6a92c355f36a admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:38.040718 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd92a0-410f-43ee-a2a0-944ae2dde83f None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:38.041933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd92a0-410f-43ee-a2a0-944ae2dde83f None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:38.042403 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd92a0-410f-43ee-a2a0-944ae2dde83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:38.117051 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-3fcd92a0-410f-43ee-a2a0-944ae2dde83f None None] MFA Rules not processed for user `fe85040c0552463ea64e2e90b0a25466`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:38.121459 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-3fcd92a0-410f-43ee-a2a0-944ae2dde83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:38.174758 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d3243-ff3d-427f-b2bb-45db1d2e7af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:38.175620 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d3243-ff3d-427f-b2bb-45db1d2e7af7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:38.175976 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d3243-ff3d-427f-b2bb-45db1d2e7af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:38.176340 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d3243-ff3d-427f-b2bb-45db1d2e7af7 admin admin] PATH_INFO: `/v3/projects/afa34b8e375c470f97ee04ad9ec271ba` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:38.180211 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3d3243-ff3d-427f-b2bb-45db1d2e7af7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afa34b8e375c470f97ee04ad9ec271ba)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:38.187991 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3d3243-ff3d-427f-b2bb-45db1d2e7af7 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:38.365318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-5d3d3243-ff3d-427f-b2bb-45db1d2e7af7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afa34b8e375c470f97ee04ad9ec271ba', 'request_id': 'req-5d3d3243-ff3d-427f-b2bb-45db1d2e7af7'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:38.389381 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 880/1756] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:38 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:38.390333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-5d3d3243-ff3d-427f-b2bb-45db1d2e7af7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afa34b8e375c470f97ee04ad9ec271ba', 'request_id': 'req-5d3d3243-ff3d-427f-b2bb-45db1d2e7af7'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:38.412548 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd92a0-410f-43ee-a2a0-944ae2dde83f None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:38.414554 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-3fcd92a0-410f-43ee-a2a0-944ae2dde83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:38.424383 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-5d3d3243-ff3d-427f-b2bb-45db1d2e7af7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afa34b8e375c470f97ee04ad9ec271ba', 'request_id': 'req-5d3d3243-ff3d-427f-b2bb-45db1d2e7af7'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:38.440865 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:38.487256 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-5d3d3243-ff3d-427f-b2bb-45db1d2e7af7 admin admin] The token cache is being invalidate because project afa34b8e375c470f97ee04ad9ec271ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:27:38.488304 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-5d3d3243-ff3d-427f-b2bb-45db1d2e7af7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:38.492821 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-5d3d3243-ff3d-427f-b2bb-45db1d2e7af7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:38.504075 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 877/1757] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:27:37 2022] DELETE /identity/v3/projects/afa34b8e375c470f97ee04ad9ec271ba => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:38.508328 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60349b-9347-4d76-8e51-fab4e05e1598 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:38.511577 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60349b-9347-4d76-8e51-fab4e05e1598 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:38.514149 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60349b-9347-4d76-8e51-fab4e05e1598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:38.602447 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-fa60349b-9347-4d76-8e51-fab4e05e1598 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:38.606543 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-fa60349b-9347-4d76-8e51-fab4e05e1598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:38.677967 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd92a0-410f-43ee-a2a0-944ae2dde83f None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:38.717401 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:38.793177 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c583146-c32f-4111-87cf-6ab2e890ef5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:38.793937 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c583146-c32f-4111-87cf-6ab2e890ef5d service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:38.794221 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c583146-c32f-4111-87cf-6ab2e890ef5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:38.794490 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c583146-c32f-4111-87cf-6ab2e890ef5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:38.800108 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-2c583146-c32f-4111-87cf-6ab2e890ef5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:38.805616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 878/1758] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:38.807410 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a926ebc-b681-4321-88fb-c100cc058cb1 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:38.807828 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a926ebc-b681-4321-88fb-c100cc058cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:38.808228 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a926ebc-b681-4321-88fb-c100cc058cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:38.832294 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-2a926ebc-b681-4321-88fb-c100cc058cb1 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:38.834681 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-2a926ebc-b681-4321-88fb-c100cc058cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:38.863815 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 879/1759] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:38.868780 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a926ebc-b681-4321-88fb-c100cc058cb1 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:38.899241 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a926ebc-b681-4321-88fb-c100cc058cb1 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:38.934560 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c395ee-adba-4583-86b8-ef797376981b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:38.935352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c395ee-adba-4583-86b8-ef797376981b admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:38.935643 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c395ee-adba-4583-86b8-ef797376981b admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:38.935940 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c395ee-adba-4583-86b8-ef797376981b admin admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:38.941991 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c583146-c32f-4111-87cf-6ab2e890ef5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:38.944247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c583146-c32f-4111-87cf-6ab2e890ef5d service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:38.945508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c395ee-adba-4583-86b8-ef797376981b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:27:38.945926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c395ee-adba-4583-86b8-ef797376981b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:38.954592 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c395ee-adba-4583-86b8-ef797376981b admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:38.975205 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 881/1760] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:38 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:27:39.011227 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c583146-c32f-4111-87cf-6ab2e890ef5d service cinder] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:39.012338 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 880/1761] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:27:39.048804 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c583146-c32f-4111-87cf-6ab2e890ef5d service cinder] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:39.067531 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1596b6-8fed-40f8-9128-e1006074253b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:39.068545 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1596b6-8fed-40f8-9128-e1006074253b admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:39.069410 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1596b6-8fed-40f8-9128-e1006074253b admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:39.071950 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1596b6-8fed-40f8-9128-e1006074253b admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:39.072770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1596b6-8fed-40f8-9128-e1006074253b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:39.074879 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1596b6-8fed-40f8-9128-e1006074253b admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:39.168346 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 882/1762] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:27:39 2022] POST /identity/v3/projects => generated 359 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:39.172346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c395ee-adba-4583-86b8-ef797376981b admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:39.191658 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c395ee-adba-4583-86b8-ef797376981b admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:39.216795 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1596b6-8fed-40f8-9128-e1006074253b admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:39.218765 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-2e1596b6-8fed-40f8-9128-e1006074253b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:39.318116 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a8c77-56b0-478f-af26-4cd1ee8e7962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:39.318116 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a8c77-56b0-478f-af26-4cd1ee8e7962 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:39.318418 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a8c77-56b0-478f-af26-4cd1ee8e7962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:39.318605 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a8c77-56b0-478f-af26-4cd1ee8e7962 admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:39.321876 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267a8c77-56b0-478f-af26-4cd1ee8e7962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:39.324390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267a8c77-56b0-478f-af26-4cd1ee8e7962 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:39.334782 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1596b6-8fed-40f8-9128-e1006074253b admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:39.345614 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97907f44-1258-4a16-b6fe-5006ec098b5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:39.346951 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97907f44-1258-4a16-b6fe-5006ec098b5e service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:39.347318 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97907f44-1258-4a16-b6fe-5006ec098b5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:39.347713 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97907f44-1258-4a16-b6fe-5006ec098b5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:39.358818 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-97907f44-1258-4a16-b6fe-5006ec098b5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:39.395305 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 881/1763] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:27:39 2022] POST /identity/v3/users => generated 280 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:39.405802 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a8c77-56b0-478f-af26-4cd1ee8e7962 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:39.458601 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a8c77-56b0-478f-af26-4cd1ee8e7962 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:39.477664 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e0acdd-5fbc-4f5d-9b0d-1a556dbd1012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:39.479446 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0acdd-5fbc-4f5d-9b0d-1a556dbd1012 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:39.480657 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0acdd-5fbc-4f5d-9b0d-1a556dbd1012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:39.480874 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0acdd-5fbc-4f5d-9b0d-1a556dbd1012 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:39.482716 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e0acdd-5fbc-4f5d-9b0d-1a556dbd1012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:39.486352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e0acdd-5fbc-4f5d-9b0d-1a556dbd1012 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:39.533510 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 882/1764] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:27:39.534599 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97907f44-1258-4a16-b6fe-5006ec098b5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:39.535033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e0acdd-5fbc-4f5d-9b0d-1a556dbd1012 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:39.536514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97907f44-1258-4a16-b6fe-5006ec098b5e service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:39.555623 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 883/1765] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:39 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:27:39.573679 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e0acdd-5fbc-4f5d-9b0d-1a556dbd1012 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:39.628151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a07bc-6a15-4408-b078-31fa43ba35a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:39.633379 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a07bc-6a15-4408-b078-31fa43ba35a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:39.633687 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a07bc-6a15-4408-b078-31fa43ba35a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:39.637535 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a07bc-6a15-4408-b078-31fa43ba35a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:39.641196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a07bc-6a15-4408-b078-31fa43ba35a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:39.647232 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a07bc-6a15-4408-b078-31fa43ba35a6 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:39.695663 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 883/1766] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:39.699872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97907f44-1258-4a16-b6fe-5006ec098b5e service cinder] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:39.718368 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97907f44-1258-4a16-b6fe-5006ec098b5e service cinder] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:39.739796 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5fde5e-ab81-4460-84d5-c3072cbc984a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:39.741249 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fde5e-ab81-4460-84d5-c3072cbc984a admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:39.741725 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fde5e-ab81-4460-84d5-c3072cbc984a admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:39.743369 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fde5e-ab81-4460-84d5-c3072cbc984a admin admin] PATH_INFO: `/v3/projects/9f1c5eb51d8e4ee69a39c26390071cf4/users/e31225884ecb40bdb69e62708ec77067/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:39.749012 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:39.789317 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5fde5e-ab81-4460-84d5-c3072cbc984a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f1c5eb51d8e4ee69a39c26390071cf4, user_id=e31225884ecb40bdb69e62708ec77067, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:39.794411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5fde5e-ab81-4460-84d5-c3072cbc984a admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:39.837263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 884/1767] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:39 2022] PUT /identity/v3/projects/9f1c5eb51d8e4ee69a39c26390071cf4/users/e31225884ecb40bdb69e62708ec77067/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:39.849256 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff31c2-a9f1-4f97-99b0-890ac035802f None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:39.849540 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff31c2-a9f1-4f97-99b0-890ac035802f None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:39.849815 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff31c2-a9f1-4f97-99b0-890ac035802f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:39.933861 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-98ff31c2-a9f1-4f97-99b0-890ac035802f None None] MFA Rules not processed for user `e31225884ecb40bdb69e62708ec77067`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:39.943512 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-98ff31c2-a9f1-4f97-99b0-890ac035802f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:40.045218 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:40.167002 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 884/1768] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:39 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:40.172203 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5fde5e-ab81-4460-84d5-c3072cbc984a admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:40.284303 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d67277-d617-45b2-b993-80f9a6a88808 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:40.284846 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d67277-d617-45b2-b993-80f9a6a88808 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:40.284846 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d67277-d617-45b2-b993-80f9a6a88808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:40.348213 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-e1d67277-d617-45b2-b993-80f9a6a88808 None None] MFA Rules not processed for user `820b9a99be574eb8b09629988c4c59c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:40.355565 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-e1d67277-d617-45b2-b993-80f9a6a88808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:40.368861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5fde5e-ab81-4460-84d5-c3072cbc984a admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:40.382173 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad0898-2c28-443a-ae72-52fcb1a2b23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:40.383733 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad0898-2c28-443a-ae72-52fcb1a2b23e admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:40.384135 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad0898-2c28-443a-ae72-52fcb1a2b23e admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:40.384387 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad0898-2c28-443a-ae72-52fcb1a2b23e admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:40.386098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad0898-2c28-443a-ae72-52fcb1a2b23e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:40.391783 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad0898-2c28-443a-ae72-52fcb1a2b23e admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:40.448050 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 885/1769] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:27:40 2022] POST /identity/v3/projects => generated 365 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:40.451646 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad0898-2c28-443a-ae72-52fcb1a2b23e admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:40.591291 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 885/1770] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:40 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:40.701897 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad0898-2c28-443a-ae72-52fcb1a2b23e admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:40.719741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb9197-c74f-47ef-90c6-4701488ccb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:40.720755 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb9197-c74f-47ef-90c6-4701488ccb3c admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:40.721002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb9197-c74f-47ef-90c6-4701488ccb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:40.721271 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb9197-c74f-47ef-90c6-4701488ccb3c admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:40.722212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb9197-c74f-47ef-90c6-4701488ccb3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:40.724236 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb9197-c74f-47ef-90c6-4701488ccb3c admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:40.779977 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 886/1771] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:27:40 2022] POST /identity/v3/users => generated 289 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:27:40.780604 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d67277-d617-45b2-b993-80f9a6a88808 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:40.802362 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d67277-d617-45b2-b993-80f9a6a88808 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:40.821200 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf18bf1-42f7-43f4-9a57-1ae728e2c2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:40.826740 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf18bf1-42f7-43f4-9a57-1ae728e2c2c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:40.827114 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf18bf1-42f7-43f4-9a57-1ae728e2c2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:40.828546 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf18bf1-42f7-43f4-9a57-1ae728e2c2c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:40.829617 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf18bf1-42f7-43f4-9a57-1ae728e2c2c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:40.833129 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf18bf1-42f7-43f4-9a57-1ae728e2c2c6 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:40.863616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 886/1772] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:40.865059 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb9197-c74f-47ef-90c6-4701488ccb3c admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:40.889029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb9197-c74f-47ef-90c6-4701488ccb3c admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:40.907052 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d1a48-ad8f-4096-8f4e-33829298db3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:40.911141 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d1a48-ad8f-4096-8f4e-33829298db3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:40.913768 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d1a48-ad8f-4096-8f4e-33829298db3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:40.914713 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d1a48-ad8f-4096-8f4e-33829298db3c admin admin] PATH_INFO: `/v3/projects/c7be68e19cc6491db4ee4a9d8221af20/users/f2a522ba78584fcf9f3e09a7ed883707/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:40.987223 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d1a48-ad8f-4096-8f4e-33829298db3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7be68e19cc6491db4ee4a9d8221af20, user_id=f2a522ba78584fcf9f3e09a7ed883707, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:40.995045 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d1a48-ad8f-4096-8f4e-33829298db3c admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:41.026673 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 887/1773] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:40 2022] PUT /identity/v3/projects/c7be68e19cc6491db4ee4a9d8221af20/users/f2a522ba78584fcf9f3e09a7ed883707/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:41.030214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf18bf1-42f7-43f4-9a57-1ae728e2c2c6 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:41.042983 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf18bf1-42f7-43f4-9a57-1ae728e2c2c6 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:41.187859 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b27fd98-602a-48ac-af9d-859aeac131db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:41.188115 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27fd98-602a-48ac-af9d-859aeac131db admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:41.188552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27fd98-602a-48ac-af9d-859aeac131db admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:41.188693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27fd98-602a-48ac-af9d-859aeac131db admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:41.189914 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b27fd98-602a-48ac-af9d-859aeac131db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:41.191551 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b27fd98-602a-48ac-af9d-859aeac131db admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:41.204454 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 887/1774] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:41.208358 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d1a48-ad8f-4096-8f4e-33829298db3c admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:41.220047 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d1a48-ad8f-4096-8f4e-33829298db3c admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:41.225570 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a06a3-b3ce-451a-bc2e-bfe3255864ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:41.226124 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a06a3-b3ce-451a-bc2e-bfe3255864ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:41.226407 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a06a3-b3ce-451a-bc2e-bfe3255864ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:41.226682 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a06a3-b3ce-451a-bc2e-bfe3255864ad admin admin] PATH_INFO: `/v3/projects/c7be68e19cc6491db4ee4a9d8221af20/users/f2a522ba78584fcf9f3e09a7ed883707/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:41.228676 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a06a3-b3ce-451a-bc2e-bfe3255864ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7be68e19cc6491db4ee4a9d8221af20, user_id=f2a522ba78584fcf9f3e09a7ed883707, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:41.232165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a06a3-b3ce-451a-bc2e-bfe3255864ad admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:41.270393 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 888/1775] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:41 2022] PUT /identity/v3/projects/c7be68e19cc6491db4ee4a9d8221af20/users/f2a522ba78584fcf9f3e09a7ed883707/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:41.282222 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17408dcd-30c2-405c-b605-1f7b3d1186b9 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:41.282873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17408dcd-30c2-405c-b605-1f7b3d1186b9 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:41.283576 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17408dcd-30c2-405c-b605-1f7b3d1186b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:41.366136 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-17408dcd-30c2-405c-b605-1f7b3d1186b9 None None] MFA Rules not processed for user `f2a522ba78584fcf9f3e09a7ed883707`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:41.367134 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-17408dcd-30c2-405c-b605-1f7b3d1186b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:41.574382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:41.605151 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26aa54-f3dc-4fcf-ac58-73c92c1fc7fe None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:41.605445 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26aa54-f3dc-4fcf-ac58-73c92c1fc7fe None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:41.605722 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26aa54-f3dc-4fcf-ac58-73c92c1fc7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:41.701859 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 888/1776] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:41 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 420 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:41.725166 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:41.786016 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-df26aa54-f3dc-4fcf-ac58-73c92c1fc7fe None None] MFA Rules not processed for user `d3538271ebfd41aabe9f3da85feb7bca`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:41.788924 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-df26aa54-f3dc-4fcf-ac58-73c92c1fc7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:41.835653 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17408dcd-30c2-405c-b605-1f7b3d1186b9 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:41.837542 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-17408dcd-30c2-405c-b605-1f7b3d1186b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:42.068690 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 889/1777] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Aug 23 04:27:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:27:42.091156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17408dcd-30c2-405c-b605-1f7b3d1186b9 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:42.103343 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc9856-e06b-459e-8f40-42016b62c6c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:42.104113 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc9856-e06b-459e-8f40-42016b62c6c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:42.104402 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc9856-e06b-459e-8f40-42016b62c6c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:42.104672 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc9856-e06b-459e-8f40-42016b62c6c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:42.124919 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc9856-e06b-459e-8f40-42016b62c6c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:42.133961 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc9856-e06b-459e-8f40-42016b62c6c6 service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:42.172435 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 889/1778] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:41 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:27:42.290507 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26aa54-f3dc-4fcf-ac58-73c92c1fc7fe None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:42.292024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-df26aa54-f3dc-4fcf-ac58-73c92c1fc7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:42.378857 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc9856-e06b-459e-8f40-42016b62c6c6 service cinder] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:42.380350 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-adbc9856-e06b-459e-8f40-42016b62c6c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:42.418177 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26aa54-f3dc-4fcf-ac58-73c92c1fc7fe None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:42.425254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb16282-0681-413f-98d3-ea47af23e420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:42.426178 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb16282-0681-413f-98d3-ea47af23e420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:42.426639 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb16282-0681-413f-98d3-ea47af23e420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:42.427021 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb16282-0681-413f-98d3-ea47af23e420 admin admin] PATH_INFO: `/v3/users/76e14e6df487455f86e69d2939676bbb` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:42.428577 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb16282-0681-413f-98d3-ea47af23e420 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76e14e6df487455f86e69d2939676bbb)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:42.431071 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb16282-0681-413f-98d3-ea47af23e420 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:42.526515 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-dbb16282-0681-413f-98d3-ea47af23e420 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76e14e6df487455f86e69d2939676bbb'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:42.527720 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-dbb16282-0681-413f-98d3-ea47af23e420 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76e14e6df487455f86e69d2939676bbb'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:42.542851 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-dbb16282-0681-413f-98d3-ea47af23e420 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76e14e6df487455f86e69d2939676bbb'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:42.574320 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-dbb16282-0681-413f-98d3-ea47af23e420 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76e14e6df487455f86e69d2939676bbb'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:42.590733 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-dbb16282-0681-413f-98d3-ea47af23e420 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76e14e6df487455f86e69d2939676bbb'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:42.608961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 890/1779] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:27:42 2022] DELETE /identity/v3/users/76e14e6df487455f86e69d2939676bbb => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:42.610455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb16282-0681-413f-98d3-ea47af23e420 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:42.612173 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-dbb16282-0681-413f-98d3-ea47af23e420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:42.641694 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:42.670587 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc9856-e06b-459e-8f40-42016b62c6c6 service cinder] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:42.748679 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb16282-0681-413f-98d3-ea47af23e420 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:42.754333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7dff3-224d-46b7-b0d1-6f6627077283 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:42.755018 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7dff3-224d-46b7-b0d1-6f6627077283 service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:42.755302 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7dff3-224d-46b7-b0d1-6f6627077283 service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:42.755577 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7dff3-224d-46b7-b0d1-6f6627077283 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:42.758092 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-68b7dff3-224d-46b7-b0d1-6f6627077283 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:42.811151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7178ec-8f5b-42d1-9aae-d8cd4d8fecf6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f1418fd211a448da1a50175dce13b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:42.813134 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7178ec-8f5b-42d1-9aae-d8cd4d8fecf6 service swift] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:42.814710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7178ec-8f5b-42d1-9aae-d8cd4d8fecf6 service swift] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:42.815053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7178ec-8f5b-42d1-9aae-d8cd4d8fecf6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:42.818760 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-cf7178ec-8f5b-42d1-9aae-d8cd4d8fecf6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:42.836866 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:42.911501 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b7dff3-224d-46b7-b0d1-6f6627077283 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:42.913002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b7dff3-224d-46b7-b0d1-6f6627077283 service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:42.937282 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 891/1780] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:42 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:42.939098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555816be-e104-4d9f-99e5-c1e25322573b None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:42.941879 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555816be-e104-4d9f-99e5-c1e25322573b None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:42.942170 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555816be-e104-4d9f-99e5-c1e25322573b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:42.974617 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7178ec-8f5b-42d1-9aae-d8cd4d8fecf6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:42.979255 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7178ec-8f5b-42d1-9aae-d8cd4d8fecf6 service swift] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:43.008931 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-555816be-e104-4d9f-99e5-c1e25322573b None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:43.015117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 890/1781] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Aug 23 04:27:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 23 04:27:43.015370 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-555816be-e104-4d9f-99e5-c1e25322573b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:43.017620 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c2b05-3551-4b95-94b4-b20e58502b18 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:43.017906 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c2b05-3551-4b95-94b4-b20e58502b18 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:43.019412 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c2b05-3551-4b95-94b4-b20e58502b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:43.068346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-7f4c2b05-3551-4b95-94b4-b20e58502b18 None None] MFA Rules not processed for user `a91872508ecd49d98bd95651412f85a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:43.075035 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-7f4c2b05-3551-4b95-94b4-b20e58502b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:43.216805 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 892/1782] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:43.242915 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555816be-e104-4d9f-99e5-c1e25322573b None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:43.246370 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-555816be-e104-4d9f-99e5-c1e25322573b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:43.261636 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 891/1783] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:43.444174 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555816be-e104-4d9f-99e5-c1e25322573b None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:43.479189 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c676897-ddb8-435e-aeda-10653c107ca1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:43.480941 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c676897-ddb8-435e-aeda-10653c107ca1 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:43.482847 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c676897-ddb8-435e-aeda-10653c107ca1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:43.483390 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c676897-ddb8-435e-aeda-10653c107ca1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:43.512193 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c676897-ddb8-435e-aeda-10653c107ca1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:43.513687 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c676897-ddb8-435e-aeda-10653c107ca1 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:43.542676 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 893/1784] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:43.662630 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c2b05-3551-4b95-94b4-b20e58502b18 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:43.664629 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-7f4c2b05-3551-4b95-94b4-b20e58502b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:43.694797 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c2b05-3551-4b95-94b4-b20e58502b18 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:43.710035 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50c05b-6be9-4647-88b4-d2fa34f82310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:43.711159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50c05b-6be9-4647-88b4-d2fa34f82310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:43.711585 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50c05b-6be9-4647-88b4-d2fa34f82310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:43.711997 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50c05b-6be9-4647-88b4-d2fa34f82310 admin admin] PATH_INFO: `/v3/users/21ad6e06e6f74753aa94eebfed82d009` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:43.715577 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50c05b-6be9-4647-88b4-d2fa34f82310 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21ad6e06e6f74753aa94eebfed82d009)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:43.717916 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50c05b-6be9-4647-88b4-d2fa34f82310 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:43.860326 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-db50c05b-6be9-4647-88b4-d2fa34f82310 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21ad6e06e6f74753aa94eebfed82d009'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:43.877158 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-db50c05b-6be9-4647-88b4-d2fa34f82310 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21ad6e06e6f74753aa94eebfed82d009'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:43.922163 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-db50c05b-6be9-4647-88b4-d2fa34f82310 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21ad6e06e6f74753aa94eebfed82d009'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:43.923280 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-db50c05b-6be9-4647-88b4-d2fa34f82310 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21ad6e06e6f74753aa94eebfed82d009'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:43.945205 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-db50c05b-6be9-4647-88b4-d2fa34f82310 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21ad6e06e6f74753aa94eebfed82d009'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:43.963141 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 892/1785] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:27:43 2022] DELETE /identity/v3/users/21ad6e06e6f74753aa94eebfed82d009 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:43.968229 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b7f3c-dc77-44a4-82a7-1ef9e109a94e None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:43.969545 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b7f3c-dc77-44a4-82a7-1ef9e109a94e None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:43.969992 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b7f3c-dc77-44a4-82a7-1ef9e109a94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:44.026811 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-2c3b7f3c-dc77-44a4-82a7-1ef9e109a94e None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:44.028801 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-2c3b7f3c-dc77-44a4-82a7-1ef9e109a94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:44.037575 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:44.283971 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 894/1786] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:44.396609 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50c05b-6be9-4647-88b4-d2fa34f82310 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:44.401453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-db50c05b-6be9-4647-88b4-d2fa34f82310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:44.523394 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:44.588717 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50c05b-6be9-4647-88b4-d2fa34f82310 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:44.593225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0caa93-b9b8-4568-898a-dff4a236e362 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:44.594143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0caa93-b9b8-4568-898a-dff4a236e362 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:44.594481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0caa93-b9b8-4568-898a-dff4a236e362 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:44.594758 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0caa93-b9b8-4568-898a-dff4a236e362 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:44.647954 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0caa93-b9b8-4568-898a-dff4a236e362 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:44.654414 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0caa93-b9b8-4568-898a-dff4a236e362 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:44.687170 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 893/1787] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:45.889809 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b7f3c-dc77-44a4-82a7-1ef9e109a94e None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:45.892074 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-2c3b7f3c-dc77-44a4-82a7-1ef9e109a94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:45.921039 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b7f3c-dc77-44a4-82a7-1ef9e109a94e None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:45.930865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac157280-ad6b-419a-8f36-b95680fe6db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:45.931578 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac157280-ad6b-419a-8f36-b95680fe6db1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:45.932013 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac157280-ad6b-419a-8f36-b95680fe6db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:45.932451 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac157280-ad6b-419a-8f36-b95680fe6db1 admin admin] PATH_INFO: `/v3/projects/197827793ebf446e95158012bfbfc0d9` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:45.934134 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac157280-ad6b-419a-8f36-b95680fe6db1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=197827793ebf446e95158012bfbfc0d9)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:45.936478 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac157280-ad6b-419a-8f36-b95680fe6db1 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:46.133479 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ac157280-ad6b-419a-8f36-b95680fe6db1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '197827793ebf446e95158012bfbfc0d9', 'request_id': 'req-ac157280-ad6b-419a-8f36-b95680fe6db1'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:46.158856 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ac157280-ad6b-419a-8f36-b95680fe6db1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '197827793ebf446e95158012bfbfc0d9', 'request_id': 'req-ac157280-ad6b-419a-8f36-b95680fe6db1'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:46.202315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ac157280-ad6b-419a-8f36-b95680fe6db1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '197827793ebf446e95158012bfbfc0d9', 'request_id': 'req-ac157280-ad6b-419a-8f36-b95680fe6db1'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:46.240352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ac157280-ad6b-419a-8f36-b95680fe6db1 admin admin] The token cache is being invalidate because project 197827793ebf446e95158012bfbfc0d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:27:46.240763 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ac157280-ad6b-419a-8f36-b95680fe6db1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:46.243294 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ac157280-ad6b-419a-8f36-b95680fe6db1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:46.245168 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 895/1788] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:27:45 2022] DELETE /identity/v3/projects/197827793ebf446e95158012bfbfc0d9 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:46.248871 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0caa93-b9b8-4568-898a-dff4a236e362 service neutron] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:46.270270 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0caa93-b9b8-4568-898a-dff4a236e362 service neutron] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:46.549367 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9dd2a-cb5a-4942-92e7-b31ed11ddb66 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:46.550090 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9dd2a-cb5a-4942-92e7-b31ed11ddb66 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:46.550604 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9dd2a-cb5a-4942-92e7-b31ed11ddb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:46.582605 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-51e9dd2a-cb5a-4942-92e7-b31ed11ddb66 None None] MFA Rules not processed for user `fdca9c48e5d740f6bda8c48072b54c0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:46.594771 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-51e9dd2a-cb5a-4942-92e7-b31ed11ddb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:46.693243 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b759b-b533-42f7-bd7e-6bf93ed1f016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:46.699372 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b759b-b533-42f7-bd7e-6bf93ed1f016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:46.702128 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b759b-b533-42f7-bd7e-6bf93ed1f016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:46.704541 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b759b-b533-42f7-bd7e-6bf93ed1f016 admin admin] PATH_INFO: `/v3/users/3aeaeacb4f964c41a21e7fa7b0e31f7f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:46.708644 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b759b-b533-42f7-bd7e-6bf93ed1f016 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aeaeacb4f964c41a21e7fa7b0e31f7f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:46.715061 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b759b-b533-42f7-bd7e-6bf93ed1f016 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:46.826373 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 896/1789] 10.222.0.47 () {58 vars in 1015 bytes} [Tue Aug 23 04:27:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:27:46.959166 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9dd2a-cb5a-4942-92e7-b31ed11ddb66 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:46.960656 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-51e9dd2a-cb5a-4942-92e7-b31ed11ddb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:46.968949 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-607b759b-b533-42f7-bd7e-6bf93ed1f016 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aeaeacb4f964c41a21e7fa7b0e31f7f'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:47.006265 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-607b759b-b533-42f7-bd7e-6bf93ed1f016 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aeaeacb4f964c41a21e7fa7b0e31f7f'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:47.017240 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9dd2a-cb5a-4942-92e7-b31ed11ddb66 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:47.041006 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9520d7-190f-4f42-ac06-dcd95122fe84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:47.044644 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9520d7-190f-4f42-ac06-dcd95122fe84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:47.045506 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9520d7-190f-4f42-ac06-dcd95122fe84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:47.045800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9520d7-190f-4f42-ac06-dcd95122fe84 admin admin] PATH_INFO: `/v3/users/e31225884ecb40bdb69e62708ec77067` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:47.052114 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9520d7-190f-4f42-ac06-dcd95122fe84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e31225884ecb40bdb69e62708ec77067)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:47.069918 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9520d7-190f-4f42-ac06-dcd95122fe84 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:47.090787 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-607b759b-b533-42f7-bd7e-6bf93ed1f016 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aeaeacb4f964c41a21e7fa7b0e31f7f'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:47.093408 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-607b759b-b533-42f7-bd7e-6bf93ed1f016 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aeaeacb4f964c41a21e7fa7b0e31f7f'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:47.124113 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-607b759b-b533-42f7-bd7e-6bf93ed1f016 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aeaeacb4f964c41a21e7fa7b0e31f7f'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:47.163246 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 894/1790] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:27:46 2022] DELETE /identity/v3/users/3aeaeacb4f964c41a21e7fa7b0e31f7f => generated 0 bytes in 915 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:47.164585 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b759b-b533-42f7-bd7e-6bf93ed1f016 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:47.180597 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-607b759b-b533-42f7-bd7e-6bf93ed1f016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:47.202212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:47.226499 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-2c9520d7-190f-4f42-ac06-dcd95122fe84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e31225884ecb40bdb69e62708ec77067'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:47.227442 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-2c9520d7-190f-4f42-ac06-dcd95122fe84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e31225884ecb40bdb69e62708ec77067'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:47.245140 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-2c9520d7-190f-4f42-ac06-dcd95122fe84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e31225884ecb40bdb69e62708ec77067'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:47.281695 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-2c9520d7-190f-4f42-ac06-dcd95122fe84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e31225884ecb40bdb69e62708ec77067'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:47.298356 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-2c9520d7-190f-4f42-ac06-dcd95122fe84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e31225884ecb40bdb69e62708ec77067'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:47.316749 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 897/1791] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:27:46 2022] DELETE /identity/v3/users/e31225884ecb40bdb69e62708ec77067 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:47.318013 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9520d7-190f-4f42-ac06-dcd95122fe84 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:47.320202 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-2c9520d7-190f-4f42-ac06-dcd95122fe84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:47.404738 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b759b-b533-42f7-bd7e-6bf93ed1f016 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:47.419392 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:47.541923 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9520d7-190f-4f42-ac06-dcd95122fe84 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:47.549299 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad3bb4-a244-4cf3-8060-dedcd928ff5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:47.550145 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad3bb4-a244-4cf3-8060-dedcd928ff5a service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:47.550454 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad3bb4-a244-4cf3-8060-dedcd928ff5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:47.550759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad3bb4-a244-4cf3-8060-dedcd928ff5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:47.553238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a3ad3bb4-a244-4cf3-8060-dedcd928ff5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:47.572133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad3bb4-a244-4cf3-8060-dedcd928ff5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:47.574082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad3bb4-a244-4cf3-8060-dedcd928ff5a service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:47.584662 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 898/1792] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:47.586173 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ead64e-a5a5-4862-b4b0-e77e8181e380 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:47.586476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ead64e-a5a5-4862-b4b0-e77e8181e380 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:47.587010 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ead64e-a5a5-4862-b4b0-e77e8181e380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:47.626060 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-49ead64e-a5a5-4862-b4b0-e77e8181e380 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:47.627998 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-49ead64e-a5a5-4862-b4b0-e77e8181e380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:47.649542 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe4333-53e9-4f15-be78-0305febaf5de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:47.650743 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe4333-53e9-4f15-be78-0305febaf5de service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:47.651648 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe4333-53e9-4f15-be78-0305febaf5de service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:47.652853 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe4333-53e9-4f15-be78-0305febaf5de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:47.670253 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe4333-53e9-4f15-be78-0305febaf5de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:47.680485 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe4333-53e9-4f15-be78-0305febaf5de service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:47.704277 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 895/1793] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:47.758653 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe4333-53e9-4f15-be78-0305febaf5de service neutron] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:47.760643 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-d3fe4333-53e9-4f15-be78-0305febaf5de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:47.833166 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 899/1794] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:47.929426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe4333-53e9-4f15-be78-0305febaf5de service neutron] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:47.936223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b659368-192f-4612-b0cb-adcfc52581f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:47.937710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b659368-192f-4612-b0cb-adcfc52581f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:47.938158 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b659368-192f-4612-b0cb-adcfc52581f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:47.938602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b659368-192f-4612-b0cb-adcfc52581f3 admin admin] PATH_INFO: `/v3/projects/8660bc7b611147d2be0cf05f802226bf` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:47.940104 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b659368-192f-4612-b0cb-adcfc52581f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8660bc7b611147d2be0cf05f802226bf)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:47.942458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b659368-192f-4612-b0cb-adcfc52581f3 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:48.006392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ead64e-a5a5-4862-b4b0-e77e8181e380 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:48.008700 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-49ead64e-a5a5-4862-b4b0-e77e8181e380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:48.055964 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ead64e-a5a5-4862-b4b0-e77e8181e380 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:48.080138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9b659368-192f-4612-b0cb-adcfc52581f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8660bc7b611147d2be0cf05f802226bf', 'request_id': 'req-9b659368-192f-4612-b0cb-adcfc52581f3'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:48.129157 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9b659368-192f-4612-b0cb-adcfc52581f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8660bc7b611147d2be0cf05f802226bf', 'request_id': 'req-9b659368-192f-4612-b0cb-adcfc52581f3'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:48.148698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9b659368-192f-4612-b0cb-adcfc52581f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8660bc7b611147d2be0cf05f802226bf', 'request_id': 'req-9b659368-192f-4612-b0cb-adcfc52581f3'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:48.189729 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9b659368-192f-4612-b0cb-adcfc52581f3 admin admin] The token cache is being invalidate because project 8660bc7b611147d2be0cf05f802226bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:27:48.190021 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9b659368-192f-4612-b0cb-adcfc52581f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:48.193201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9b659368-192f-4612-b0cb-adcfc52581f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:48.193800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5109b52-15c7-48db-bb1c-19ac3d0db3ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:48.194440 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5109b52-15c7-48db-bb1c-19ac3d0db3ff service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:48.194720 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5109b52-15c7-48db-bb1c-19ac3d0db3ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:48.195094 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5109b52-15c7-48db-bb1c-19ac3d0db3ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:48.197580 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a5109b52-15c7-48db-bb1c-19ac3d0db3ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:48.198473 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 896/1795] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:27:47 2022] DELETE /identity/v3/projects/8660bc7b611147d2be0cf05f802226bf => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:27:48.204145 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b659368-192f-4612-b0cb-adcfc52581f3 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:48.208702 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-9b659368-192f-4612-b0cb-adcfc52581f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:48.215462 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:48.317990 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5109b52-15c7-48db-bb1c-19ac3d0db3ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:48.319709 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5109b52-15c7-48db-bb1c-19ac3d0db3ff service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:48.347589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:48.350940 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 900/1796] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:27:48.424928 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b659368-192f-4612-b0cb-adcfc52581f3 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:48.436606 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb922d9-2e95-4b82-bc7e-ccff4b7e70de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:48.438439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb922d9-2e95-4b82-bc7e-ccff4b7e70de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:48.438935 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb922d9-2e95-4b82-bc7e-ccff4b7e70de admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:48.439425 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb922d9-2e95-4b82-bc7e-ccff4b7e70de admin admin] PATH_INFO: `/v3/users/fe85040c0552463ea64e2e90b0a25466` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:48.443899 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb922d9-2e95-4b82-bc7e-ccff4b7e70de admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe85040c0552463ea64e2e90b0a25466)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:48.446203 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb922d9-2e95-4b82-bc7e-ccff4b7e70de admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:48.573338 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-2bb922d9-2e95-4b82-bc7e-ccff4b7e70de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe85040c0552463ea64e2e90b0a25466'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:48.588329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-2bb922d9-2e95-4b82-bc7e-ccff4b7e70de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe85040c0552463ea64e2e90b0a25466'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:48.609941 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-2bb922d9-2e95-4b82-bc7e-ccff4b7e70de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe85040c0552463ea64e2e90b0a25466'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:48.611202 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-2bb922d9-2e95-4b82-bc7e-ccff4b7e70de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe85040c0552463ea64e2e90b0a25466'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:48.636825 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-2bb922d9-2e95-4b82-bc7e-ccff4b7e70de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe85040c0552463ea64e2e90b0a25466'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:48.665748 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 897/1797] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:27:48 2022] DELETE /identity/v3/users/fe85040c0552463ea64e2e90b0a25466 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:51.584467 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5109b52-15c7-48db-bb1c-19ac3d0db3ff service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:51.589754 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a5109b52-15c7-48db-bb1c-19ac3d0db3ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:51.638069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5109b52-15c7-48db-bb1c-19ac3d0db3ff service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:51.866431 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b136af-c237-48d1-98fe-a9a811f936a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:51.870082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b136af-c237-48d1-98fe-a9a811f936a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:51.871061 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b136af-c237-48d1-98fe-a9a811f936a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:51.872121 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b136af-c237-48d1-98fe-a9a811f936a3 admin admin] PATH_INFO: `/v3/projects/9f1c5eb51d8e4ee69a39c26390071cf4` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:51.875727 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b136af-c237-48d1-98fe-a9a811f936a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f1c5eb51d8e4ee69a39c26390071cf4)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:51.884595 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b136af-c237-48d1-98fe-a9a811f936a3 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:51.932356 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb922d9-2e95-4b82-bc7e-ccff4b7e70de admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:51.933893 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-2bb922d9-2e95-4b82-bc7e-ccff4b7e70de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:51.962366 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb922d9-2e95-4b82-bc7e-ccff4b7e70de admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:51.972380 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9f6b1-7180-48fb-8e15-9bb8fc3b9ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:51.974196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9f6b1-7180-48fb-8e15-9bb8fc3b9ac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:51.974487 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9f6b1-7180-48fb-8e15-9bb8fc3b9ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:51.974771 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9f6b1-7180-48fb-8e15-9bb8fc3b9ac2 admin admin] PATH_INFO: `/v3/projects/963387eedd004e16a27e452a06fa73b4` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:51.982544 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a9f6b1-7180-48fb-8e15-9bb8fc3b9ac2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=963387eedd004e16a27e452a06fa73b4)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:51.988916 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a9f6b1-7180-48fb-8e15-9bb8fc3b9ac2 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:52.014371 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-87b136af-c237-48d1-98fe-a9a811f936a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f1c5eb51d8e4ee69a39c26390071cf4', 'request_id': 'req-87b136af-c237-48d1-98fe-a9a811f936a3'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:52.028236 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-87b136af-c237-48d1-98fe-a9a811f936a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f1c5eb51d8e4ee69a39c26390071cf4', 'request_id': 'req-87b136af-c237-48d1-98fe-a9a811f936a3'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:52.043409 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-87b136af-c237-48d1-98fe-a9a811f936a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f1c5eb51d8e4ee69a39c26390071cf4', 'request_id': 'req-87b136af-c237-48d1-98fe-a9a811f936a3'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:52.068324 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-87b136af-c237-48d1-98fe-a9a811f936a3 admin admin] The token cache is being invalidate because project 9f1c5eb51d8e4ee69a39c26390071cf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:27:52.069014 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-87b136af-c237-48d1-98fe-a9a811f936a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:52.069895 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-87b136af-c237-48d1-98fe-a9a811f936a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:52.075353 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 901/1798] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:27:51 2022] DELETE /identity/v3/projects/9f1c5eb51d8e4ee69a39c26390071cf4 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:52.076708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b136af-c237-48d1-98fe-a9a811f936a3 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:52.078951 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-87b136af-c237-48d1-98fe-a9a811f936a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:52.132050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a0a9f6b1-7180-48fb-8e15-9bb8fc3b9ac2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '963387eedd004e16a27e452a06fa73b4', 'request_id': 'req-a0a9f6b1-7180-48fb-8e15-9bb8fc3b9ac2'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:52.140518 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a0a9f6b1-7180-48fb-8e15-9bb8fc3b9ac2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '963387eedd004e16a27e452a06fa73b4', 'request_id': 'req-a0a9f6b1-7180-48fb-8e15-9bb8fc3b9ac2'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:52.151122 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a0a9f6b1-7180-48fb-8e15-9bb8fc3b9ac2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '963387eedd004e16a27e452a06fa73b4', 'request_id': 'req-a0a9f6b1-7180-48fb-8e15-9bb8fc3b9ac2'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:52.158234 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:52.200143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a0a9f6b1-7180-48fb-8e15-9bb8fc3b9ac2 admin admin] The token cache is being invalidate because project 963387eedd004e16a27e452a06fa73b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:27:52.200534 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a0a9f6b1-7180-48fb-8e15-9bb8fc3b9ac2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:52.205469 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a0a9f6b1-7180-48fb-8e15-9bb8fc3b9ac2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:52.210895 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 898/1799] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:27:51 2022] DELETE /identity/v3/projects/963387eedd004e16a27e452a06fa73b4 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:52.212869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f84b4-09ec-4c65-95c2-5a8607459be0 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:52.212869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f84b4-09ec-4c65-95c2-5a8607459be0 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:52.213139 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f84b4-09ec-4c65-95c2-5a8607459be0 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:52.214939 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 899/1800] 10.222.0.18 () {54 vars in 921 bytes} [Tue Aug 23 04:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:27:52.215939 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdb8ec-f827-4f53-9e68-985c861ca739 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:52.216219 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdb8ec-f827-4f53-9e68-985c861ca739 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:52.216493 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdb8ec-f827-4f53-9e68-985c861ca739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:52.228396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b136af-c237-48d1-98fe-a9a811f936a3 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:52.272610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-7efdb8ec-f827-4f53-9e68-985c861ca739 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:52.274032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-7efdb8ec-f827-4f53-9e68-985c861ca739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:52.356924 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb15dd2-60e0-437c-b067-105ab3cf5845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:52.358095 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb15dd2-60e0-437c-b067-105ab3cf5845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:52.358644 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb15dd2-60e0-437c-b067-105ab3cf5845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:52.359639 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb15dd2-60e0-437c-b067-105ab3cf5845 admin admin] PATH_INFO: `/v3/users/f2a522ba78584fcf9f3e09a7ed883707` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:52.361157 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb15dd2-60e0-437c-b067-105ab3cf5845 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2a522ba78584fcf9f3e09a7ed883707)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:52.364309 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb15dd2-60e0-437c-b067-105ab3cf5845 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:52.382892 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:52.456447 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 900/1801] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:52.456447 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcc6ef-c57c-4053-a569-1e95684e42af None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:52.456447 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcc6ef-c57c-4053-a569-1e95684e42af None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:52.456447 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcc6ef-c57c-4053-a569-1e95684e42af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:52.487434 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-1fb15dd2-60e0-437c-b067-105ab3cf5845 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2a522ba78584fcf9f3e09a7ed883707'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:52.492316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-1fb15dd2-60e0-437c-b067-105ab3cf5845 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2a522ba78584fcf9f3e09a7ed883707'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:52.509035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-1fb15dd2-60e0-437c-b067-105ab3cf5845 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2a522ba78584fcf9f3e09a7ed883707'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:52.513163 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-09dcc6ef-c57c-4053-a569-1e95684e42af None None] MFA Rules not processed for user `cd7b11f479a54adc8d9b8641859b07a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:52.514626 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-09dcc6ef-c57c-4053-a569-1e95684e42af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:52.537993 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-1fb15dd2-60e0-437c-b067-105ab3cf5845 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2a522ba78584fcf9f3e09a7ed883707'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:52.560244 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-1fb15dd2-60e0-437c-b067-105ab3cf5845 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2a522ba78584fcf9f3e09a7ed883707'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:52.577071 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 902/1802] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:27:52 2022] DELETE /identity/v3/users/f2a522ba78584fcf9f3e09a7ed883707 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:52.577071 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b381377-8d0c-4ca2-8495-0264febeb697 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:52.577071 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b381377-8d0c-4ca2-8495-0264febeb697 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:52.578947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b381377-8d0c-4ca2-8495-0264febeb697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:52.637002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-8b381377-8d0c-4ca2-8495-0264febeb697 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:52.638798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-8b381377-8d0c-4ca2-8495-0264febeb697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:52.722307 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 901/1803] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Aug 23 04:27:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:52.887747 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 903/1804] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:52.893234 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dcc6ef-c57c-4053-a569-1e95684e42af None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:52.914666 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dcc6ef-c57c-4053-a569-1e95684e42af None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:52.930122 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3c2f9-0009-449e-a321-9f4534e36e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:52.930803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3c2f9-0009-449e-a321-9f4534e36e33 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:52.931455 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3c2f9-0009-449e-a321-9f4534e36e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:52.931455 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3c2f9-0009-449e-a321-9f4534e36e33 admin admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:52.932612 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec3c2f9-0009-449e-a321-9f4534e36e33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:27:52.933212 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec3c2f9-0009-449e-a321-9f4534e36e33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:52.934827 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec3c2f9-0009-449e-a321-9f4534e36e33 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:52.973824 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b381377-8d0c-4ca2-8495-0264febeb697 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:52.976314 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 902/1805] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:27:52.982442 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:52.993783 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b381377-8d0c-4ca2-8495-0264febeb697 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:53.007750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a307bc50-3d52-4af9-99e6-996856473475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:53.008562 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307bc50-3d52-4af9-99e6-996856473475 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:53.008961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307bc50-3d52-4af9-99e6-996856473475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:53.009309 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307bc50-3d52-4af9-99e6-996856473475 admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:53.010132 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a307bc50-3d52-4af9-99e6-996856473475 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:53.013498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a307bc50-3d52-4af9-99e6-996856473475 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:53.128082 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3c2f9-0009-449e-a321-9f4534e36e33 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:53.128619 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 904/1806] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:27:52 2022] POST /identity/v3/projects => generated 387 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:27:53.137285 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a307bc50-3d52-4af9-99e6-996856473475 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:53.139501 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a307bc50-3d52-4af9-99e6-996856473475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:53.141538 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3c2f9-0009-449e-a321-9f4534e36e33 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:53.312584 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a307bc50-3d52-4af9-99e6-996856473475 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:53.322528 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60765c8d-d695-4dc4-b4fc-7683999581a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:53.323546 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60765c8d-d695-4dc4-b4fc-7683999581a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:53.324539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60765c8d-d695-4dc4-b4fc-7683999581a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:53.325056 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60765c8d-d695-4dc4-b4fc-7683999581a4 admin admin] PATH_INFO: `/v3/projects/7b5c201ac7424cc2a9aeea2268c30f47` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:53.327869 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60765c8d-d695-4dc4-b4fc-7683999581a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b5c201ac7424cc2a9aeea2268c30f47)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:53.330677 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60765c8d-d695-4dc4-b4fc-7683999581a4 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:53.333893 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889771f2-8fad-4822-a32d-d94604c2ab57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:53.334810 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889771f2-8fad-4822-a32d-d94604c2ab57 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:53.335311 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889771f2-8fad-4822-a32d-d94604c2ab57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:53.335781 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889771f2-8fad-4822-a32d-d94604c2ab57 admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:53.336642 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889771f2-8fad-4822-a32d-d94604c2ab57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:53.340111 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889771f2-8fad-4822-a32d-d94604c2ab57 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:53.381311 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 903/1807] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:27:53 2022] POST /identity/v3/users => generated 294 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:53.386899 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889771f2-8fad-4822-a32d-d94604c2ab57 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:53.415653 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889771f2-8fad-4822-a32d-d94604c2ab57 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:53.454470 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-60765c8d-d695-4dc4-b4fc-7683999581a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b5c201ac7424cc2a9aeea2268c30f47', 'request_id': 'req-60765c8d-d695-4dc4-b4fc-7683999581a4'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:53.469254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-60765c8d-d695-4dc4-b4fc-7683999581a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b5c201ac7424cc2a9aeea2268c30f47', 'request_id': 'req-60765c8d-d695-4dc4-b4fc-7683999581a4'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:53.482284 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-60765c8d-d695-4dc4-b4fc-7683999581a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b5c201ac7424cc2a9aeea2268c30f47', 'request_id': 'req-60765c8d-d695-4dc4-b4fc-7683999581a4'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:53.528675 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-60765c8d-d695-4dc4-b4fc-7683999581a4 admin admin] The token cache is being invalidate because project 7b5c201ac7424cc2a9aeea2268c30f47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:27:53.532728 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-60765c8d-d695-4dc4-b4fc-7683999581a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:53.536434 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-60765c8d-d695-4dc4-b4fc-7683999581a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:53.544987 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 905/1808] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:27:53 2022] DELETE /identity/v3/projects/7b5c201ac7424cc2a9aeea2268c30f47 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:53.548650 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf29cf-c430-4b91-be98-81c225b42afd None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:53.551098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf29cf-c430-4b91-be98-81c225b42afd None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:53.551392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf29cf-c430-4b91-be98-81c225b42afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:53.556861 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:53.577507 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bf922-580e-4c73-8fa7-ea0b2699cd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:53.578454 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bf922-580e-4c73-8fa7-ea0b2699cd22 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:53.578713 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bf922-580e-4c73-8fa7-ea0b2699cd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:53.579174 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bf922-580e-4c73-8fa7-ea0b2699cd22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:53.579939 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564bf922-580e-4c73-8fa7-ea0b2699cd22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:53.583063 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564bf922-580e-4c73-8fa7-ea0b2699cd22 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:53.602278 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 904/1809] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:53.603313 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bf922-580e-4c73-8fa7-ea0b2699cd22 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:53.605330 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-564bf922-580e-4c73-8fa7-ea0b2699cd22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:53.620816 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bf922-580e-4c73-8fa7-ea0b2699cd22 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:53.627995 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b6852-aa4e-4120-b4dc-68456f3467ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:53.628627 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b6852-aa4e-4120-b4dc-68456f3467ca admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:53.628900 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b6852-aa4e-4120-b4dc-68456f3467ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:53.629169 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b6852-aa4e-4120-b4dc-68456f3467ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:53.629823 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33b6852-aa4e-4120-b4dc-68456f3467ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:53.631416 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33b6852-aa4e-4120-b4dc-68456f3467ca admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:53.640940 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-8abf29cf-c430-4b91-be98-81c225b42afd None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:53.642221 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-8abf29cf-c430-4b91-be98-81c225b42afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:53.653162 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 905/1810] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:53.654454 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b6852-aa4e-4120-b4dc-68456f3467ca admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:53.667670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 906/1811] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:53.669452 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adef1b6-415b-4439-8f8b-12b775d7d044 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:53.669908 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adef1b6-415b-4439-8f8b-12b775d7d044 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:53.670404 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adef1b6-415b-4439-8f8b-12b775d7d044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:53.697421 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b6852-aa4e-4120-b4dc-68456f3467ca admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:53.702607 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dceaa58-1476-426b-bf88-b591c736a2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:53.703327 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dceaa58-1476-426b-bf88-b591c736a2f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:53.703610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dceaa58-1476-426b-bf88-b591c736a2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:53.704010 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-4adef1b6-415b-4439-8f8b-12b775d7d044 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:53.704127 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dceaa58-1476-426b-bf88-b591c736a2f7 admin admin] PATH_INFO: `/v3/projects/44d5bb381bb444a98655392977ec46a9/users/7f33544ca4da4f509d057aea6275992f/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:53.706505 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-4adef1b6-415b-4439-8f8b-12b775d7d044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:53.727638 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 907/1812] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:53.730609 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adef1b6-415b-4439-8f8b-12b775d7d044 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:53.747021 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dceaa58-1476-426b-bf88-b591c736a2f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44d5bb381bb444a98655392977ec46a9, user_id=7f33544ca4da4f509d057aea6275992f, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:53.750525 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dceaa58-1476-426b-bf88-b591c736a2f7 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:53.755883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adef1b6-415b-4439-8f8b-12b775d7d044 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:53.767588 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b5f57-1d88-4632-a9f3-4deb3a9c16da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:53.769020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b5f57-1d88-4632-a9f3-4deb3a9c16da admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:53.769570 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b5f57-1d88-4632-a9f3-4deb3a9c16da admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:53.770032 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b5f57-1d88-4632-a9f3-4deb3a9c16da admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:53.771713 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b5f57-1d88-4632-a9f3-4deb3a9c16da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:27:53.772336 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b5f57-1d88-4632-a9f3-4deb3a9c16da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:53.775227 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b5f57-1d88-4632-a9f3-4deb3a9c16da admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:53.793888 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 906/1813] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:53 2022] PUT /identity/v3/projects/44d5bb381bb444a98655392977ec46a9/users/7f33544ca4da4f509d057aea6275992f/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:53.795863 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfae15-254b-42d7-bead-305411ba3867 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:53.796182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfae15-254b-42d7-bead-305411ba3867 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:53.796464 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfae15-254b-42d7-bead-305411ba3867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:53.809773 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 908/1814] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:27:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:27:53.811662 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b5f57-1d88-4632-a9f3-4deb3a9c16da admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:53.824850 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b5f57-1d88-4632-a9f3-4deb3a9c16da admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:53.887632 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-6fdfae15-254b-42d7-bead-305411ba3867 None None] MFA Rules not processed for user `7f33544ca4da4f509d057aea6275992f`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:53.891049 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-6fdfae15-254b-42d7-bead-305411ba3867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:53.894794 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:53.921211 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c0a6f-30b9-4428-877e-707cc88a70a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:53.923261 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c0a6f-30b9-4428-877e-707cc88a70a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:53.924873 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c0a6f-30b9-4428-877e-707cc88a70a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:53.925372 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c0a6f-30b9-4428-877e-707cc88a70a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:53.926454 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037c0a6f-30b9-4428-877e-707cc88a70a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:53.931003 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037c0a6f-30b9-4428-877e-707cc88a70a0 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:54.001697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 907/1815] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:53 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:27:54.002617 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdfae15-254b-42d7-bead-305411ba3867 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:54.004033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-6fdfae15-254b-42d7-bead-305411ba3867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:54.014102 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 909/1816] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:27:53 2022] POST /identity/v3/projects => generated 375 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:54.015731 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c0a6f-30b9-4428-877e-707cc88a70a0 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:54.090380 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdfae15-254b-42d7-bead-305411ba3867 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:54.136415 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:54.152849 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c0a6f-30b9-4428-877e-707cc88a70a0 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:54.160188 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a974b-eda8-40aa-98ca-e74efe63e4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:54.160819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a974b-eda8-40aa-98ca-e74efe63e4b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:54.161188 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a974b-eda8-40aa-98ca-e74efe63e4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:54.161462 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a974b-eda8-40aa-98ca-e74efe63e4b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:54.162342 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35a974b-eda8-40aa-98ca-e74efe63e4b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:54.164974 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35a974b-eda8-40aa-98ca-e74efe63e4b0 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:54.174604 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cef6f-2d79-4964-9992-b6a6494721f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:54.179612 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cef6f-2d79-4964-9992-b6a6494721f7 service nova] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:54.180013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cef6f-2d79-4964-9992-b6a6494721f7 service nova] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:54.180335 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cef6f-2d79-4964-9992-b6a6494721f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:54.185112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-1d6cef6f-2d79-4964-9992-b6a6494721f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:54.206725 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6cef6f-2d79-4964-9992-b6a6494721f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:54.215783 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6cef6f-2d79-4964-9992-b6a6494721f7 service nova] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:54.240196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 910/1817] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:27:54 2022] POST /identity/v3/projects => generated 387 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:54.240610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 908/1818] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:54.241864 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cef6f-2d79-4964-9992-b6a6494721f7 service nova] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:54.242507 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a974b-eda8-40aa-98ca-e74efe63e4b0 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:54.266801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cef6f-2d79-4964-9992-b6a6494721f7 service nova] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:54.352028 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a974b-eda8-40aa-98ca-e74efe63e4b0 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:54.364659 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37e2fc-e48f-4464-b19a-f121c9965753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:54.366704 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37e2fc-e48f-4464-b19a-f121c9965753 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:54.367201 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37e2fc-e48f-4464-b19a-f121c9965753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:54.367930 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37e2fc-e48f-4464-b19a-f121c9965753 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:54.369107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d37e2fc-e48f-4464-b19a-f121c9965753 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:54.373002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d37e2fc-e48f-4464-b19a-f121c9965753 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:54.418468 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 911/1819] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:27:54 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:54.419564 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37e2fc-e48f-4464-b19a-f121c9965753 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:54.444542 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240407f7-fd7b-4b4b-a91a-2955d11cd333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:54.444542 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240407f7-fd7b-4b4b-a91a-2955d11cd333 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:54.444542 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240407f7-fd7b-4b4b-a91a-2955d11cd333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:54.444542 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240407f7-fd7b-4b4b-a91a-2955d11cd333 admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:54.444887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240407f7-fd7b-4b4b-a91a-2955d11cd333 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:54.448333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240407f7-fd7b-4b4b-a91a-2955d11cd333 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:54.489732 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37e2fc-e48f-4464-b19a-f121c9965753 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:54.495855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 909/1820] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:27:54 2022] POST /identity/v3/users => generated 300 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:54.496992 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240407f7-fd7b-4b4b-a91a-2955d11cd333 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:54.509654 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac17bf7-0d94-432a-a3ab-c6b72e3b1486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:54.511399 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac17bf7-0d94-432a-a3ab-c6b72e3b1486 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:54.512572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac17bf7-0d94-432a-a3ab-c6b72e3b1486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:54.514124 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac17bf7-0d94-432a-a3ab-c6b72e3b1486 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:54.514890 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac17bf7-0d94-432a-a3ab-c6b72e3b1486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:54.523378 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240407f7-fd7b-4b4b-a91a-2955d11cd333 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:54.525668 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac17bf7-0d94-432a-a3ab-c6b72e3b1486 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:54.535249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726044f-a036-458e-a7ed-9c793a86dd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:54.536264 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726044f-a036-458e-a7ed-9c793a86dd41 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:54.536445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726044f-a036-458e-a7ed-9c793a86dd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:54.536907 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726044f-a036-458e-a7ed-9c793a86dd41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:54.537623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d726044f-a036-458e-a7ed-9c793a86dd41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:54.539368 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d726044f-a036-458e-a7ed-9c793a86dd41 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:54.555084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 910/1821] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:54.560947 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726044f-a036-458e-a7ed-9c793a86dd41 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:54.563307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 912/1822] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:54.569073 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac17bf7-0d94-432a-a3ab-c6b72e3b1486 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:54.591050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726044f-a036-458e-a7ed-9c793a86dd41 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:54.601720 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbfd5a5-980b-41db-b953-f104b83b98e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:54.602779 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfd5a5-980b-41db-b953-f104b83b98e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:54.603342 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfd5a5-980b-41db-b953-f104b83b98e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:54.603829 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfd5a5-980b-41db-b953-f104b83b98e6 admin admin] PATH_INFO: `/v3/projects/f1beb00eb94d468ebecc0658145fa514/users/96d3865132b7448c88efdfb7c403c2e9/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:54.606239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac17bf7-0d94-432a-a3ab-c6b72e3b1486 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:54.623480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbfd5a5-980b-41db-b953-f104b83b98e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1beb00eb94d468ebecc0658145fa514, user_id=96d3865132b7448c88efdfb7c403c2e9, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:54.629577 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbfd5a5-980b-41db-b953-f104b83b98e6 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:54.637034 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a4281-d4e8-4896-9634-50d1b19ff971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:54.637969 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a4281-d4e8-4896-9634-50d1b19ff971 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:54.638281 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a4281-d4e8-4896-9634-50d1b19ff971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:54.638588 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a4281-d4e8-4896-9634-50d1b19ff971 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:54.639799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a4281-d4e8-4896-9634-50d1b19ff971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:54.644200 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a4281-d4e8-4896-9634-50d1b19ff971 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:54.653771 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 911/1823] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:54 2022] PUT /identity/v3/projects/f1beb00eb94d468ebecc0658145fa514/users/96d3865132b7448c88efdfb7c403c2e9/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:54.658396 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbfd5a5-980b-41db-b953-f104b83b98e6 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:54.680115 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 913/1824] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:54.684157 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a4281-d4e8-4896-9634-50d1b19ff971 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:54.710791 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a4281-d4e8-4896-9634-50d1b19ff971 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:54.758187 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbfd5a5-980b-41db-b953-f104b83b98e6 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:54.760858 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:54.764922 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b0c7c-c10f-4a19-80d0-9cc8136c31b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:54.766031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b0c7c-c10f-4a19-80d0-9cc8136c31b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:54.766505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b0c7c-c10f-4a19-80d0-9cc8136c31b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:54.767359 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b0c7c-c10f-4a19-80d0-9cc8136c31b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:54.768411 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80b0c7c-c10f-4a19-80d0-9cc8136c31b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:54.770743 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80b0c7c-c10f-4a19-80d0-9cc8136c31b5 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:54.784974 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 912/1825] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:54.788736 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b0c7c-c10f-4a19-80d0-9cc8136c31b5 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:54.805117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b0c7c-c10f-4a19-80d0-9cc8136c31b5 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:54.812672 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e093b9-17eb-4922-b677-27a793c63fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:54.813796 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e093b9-17eb-4922-b677-27a793c63fc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:54.814841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e093b9-17eb-4922-b677-27a793c63fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:54.815618 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e093b9-17eb-4922-b677-27a793c63fc7 admin admin] PATH_INFO: `/v3/projects/f1beb00eb94d468ebecc0658145fa514/users/96d3865132b7448c88efdfb7c403c2e9/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:54.818579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e093b9-17eb-4922-b677-27a793c63fc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1beb00eb94d468ebecc0658145fa514, user_id=96d3865132b7448c88efdfb7c403c2e9, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:54.822033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e093b9-17eb-4922-b677-27a793c63fc7 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:54.839227 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 913/1826] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:54 2022] PUT /identity/v3/projects/f1beb00eb94d468ebecc0658145fa514/users/96d3865132b7448c88efdfb7c403c2e9/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:54.839643 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb2c40-3718-4933-8dff-61c837e376c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:54.840259 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb2c40-3718-4933-8dff-61c837e376c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:54.840540 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb2c40-3718-4933-8dff-61c837e376c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:54.840813 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb2c40-3718-4933-8dff-61c837e376c1 admin admin] PATH_INFO: `/v3/projects/48f4b081193e41cfb1f483fb8d18c045/users/b8a5935105e0406bb1e690096b5979d2/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:54.852656 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da119254-bfde-4675-bd69-2d2535877b7b None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:54.855000 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da119254-bfde-4675-bd69-2d2535877b7b None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:54.855482 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da119254-bfde-4675-bd69-2d2535877b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:54.873370 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb2c40-3718-4933-8dff-61c837e376c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48f4b081193e41cfb1f483fb8d18c045, user_id=b8a5935105e0406bb1e690096b5979d2, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:54.876177 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb2c40-3718-4933-8dff-61c837e376c1 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:54.882983 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:54.924922 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 914/1827] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:54 2022] PUT /identity/v3/projects/48f4b081193e41cfb1f483fb8d18c045/users/b8a5935105e0406bb1e690096b5979d2/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:54.946564 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-da119254-bfde-4675-bd69-2d2535877b7b None None] MFA Rules not processed for user `96d3865132b7448c88efdfb7c403c2e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:54.946827 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b09cd-5300-42fd-a0a2-6b8bd3121186 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:54.947114 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b09cd-5300-42fd-a0a2-6b8bd3121186 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:54.947554 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b09cd-5300-42fd-a0a2-6b8bd3121186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:54.950742 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-da119254-bfde-4675-bd69-2d2535877b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:55.045390 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-bd3b09cd-5300-42fd-a0a2-6b8bd3121186 None None] MFA Rules not processed for user `b8a5935105e0406bb1e690096b5979d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:55.047494 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-bd3b09cd-5300-42fd-a0a2-6b8bd3121186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:55.151934 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 914/1828] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:54 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:55.165154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da119254-bfde-4675-bd69-2d2535877b7b None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:55.166996 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-da119254-bfde-4675-bd69-2d2535877b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:55.208182 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 915/1829] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:55.213072 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b09cd-5300-42fd-a0a2-6b8bd3121186 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:55.338840 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da119254-bfde-4675-bd69-2d2535877b7b None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:55.352696 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d69177-e0cf-4fa6-b6c1-68ec24a7b460 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:55.354657 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d69177-e0cf-4fa6-b6c1-68ec24a7b460 service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:55.357103 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d69177-e0cf-4fa6-b6c1-68ec24a7b460 service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:55.357504 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d69177-e0cf-4fa6-b6c1-68ec24a7b460 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:55.380335 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d69177-e0cf-4fa6-b6c1-68ec24a7b460 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:55.382074 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d69177-e0cf-4fa6-b6c1-68ec24a7b460 service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:55.409142 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 915/1830] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:55 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:27:55.425246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b09cd-5300-42fd-a0a2-6b8bd3121186 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:55.436737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0fe89-4ea7-4fde-b1cc-bdf77842a8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:55.437729 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0fe89-4ea7-4fde-b1cc-bdf77842a8af admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:55.441424 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0fe89-4ea7-4fde-b1cc-bdf77842a8af admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:55.441723 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0fe89-4ea7-4fde-b1cc-bdf77842a8af admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:55.444755 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0fe89-4ea7-4fde-b1cc-bdf77842a8af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:55.449721 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0fe89-4ea7-4fde-b1cc-bdf77842a8af admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:55.519526 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 916/1831] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:27:55 2022] POST /identity/v3/projects => generated 375 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:55.523147 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d69177-e0cf-4fa6-b6c1-68ec24a7b460 service cinder] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:55.546175 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d69177-e0cf-4fa6-b6c1-68ec24a7b460 service cinder] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:55.639449 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6784c3-53c6-492f-8c03-8830c02a9d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:55.640748 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6784c3-53c6-492f-8c03-8830c02a9d5d admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:55.641506 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6784c3-53c6-492f-8c03-8830c02a9d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:55.641906 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6784c3-53c6-492f-8c03-8830c02a9d5d admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:55.642881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6784c3-53c6-492f-8c03-8830c02a9d5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:55.646239 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6784c3-53c6-492f-8c03-8830c02a9d5d admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:55.677358 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 916/1832] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:27:55 2022] POST /identity/v3/users => generated 294 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:55.681266 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0fe89-4ea7-4fde-b1cc-bdf77842a8af admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:55.694307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0fe89-4ea7-4fde-b1cc-bdf77842a8af admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:55.712044 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eedc874-fefb-4d94-9a92-fd67246c44b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:55.713106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedc874-fefb-4d94-9a92-fd67246c44b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:55.713388 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedc874-fefb-4d94-9a92-fd67246c44b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:55.713664 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedc874-fefb-4d94-9a92-fd67246c44b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:55.714358 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eedc874-fefb-4d94-9a92-fd67246c44b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:55.716883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eedc874-fefb-4d94-9a92-fd67246c44b2 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:55.751530 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 917/1833] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:27:55.755839 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6784c3-53c6-492f-8c03-8830c02a9d5d admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:55.770022 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6784c3-53c6-492f-8c03-8830c02a9d5d admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:55.795954 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f421a0-5bdb-4b3a-b00c-0f72e6c21984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:55.797842 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f421a0-5bdb-4b3a-b00c-0f72e6c21984 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:55.797842 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f421a0-5bdb-4b3a-b00c-0f72e6c21984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:55.797842 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f421a0-5bdb-4b3a-b00c-0f72e6c21984 admin admin] PATH_INFO: `/v3/projects/b7c4b94378ee4023a95540195c38b9e2/users/f1646b8caaa343f5adabd84982f4bad9/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:55.865191 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f421a0-5bdb-4b3a-b00c-0f72e6c21984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7c4b94378ee4023a95540195c38b9e2, user_id=f1646b8caaa343f5adabd84982f4bad9, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:55.868819 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f421a0-5bdb-4b3a-b00c-0f72e6c21984 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:55.884600 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:55.904301 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 917/1834] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:55 2022] PUT /identity/v3/projects/b7c4b94378ee4023a95540195c38b9e2/users/f1646b8caaa343f5adabd84982f4bad9/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:55.905328 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eedc874-fefb-4d94-9a92-fd67246c44b2 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:55.915946 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eedc874-fefb-4d94-9a92-fd67246c44b2 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:55.934974 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:56.014262 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc683e02-735d-4e98-8044-3b0284c463e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:56.014831 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc683e02-735d-4e98-8044-3b0284c463e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:56.015107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc683e02-735d-4e98-8044-3b0284c463e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:56.015751 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc683e02-735d-4e98-8044-3b0284c463e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:56.016548 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc683e02-735d-4e98-8044-3b0284c463e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:56.018947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc683e02-735d-4e98-8044-3b0284c463e0 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:56.048224 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 918/1835] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:56.049603 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f421a0-5bdb-4b3a-b00c-0f72e6c21984 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:56.067806 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f421a0-5bdb-4b3a-b00c-0f72e6c21984 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:56.081565 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e596a-34e8-4851-8970-48e08ace3ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:56.082305 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e596a-34e8-4851-8970-48e08ace3ff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:56.082606 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e596a-34e8-4851-8970-48e08ace3ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:56.082904 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e596a-34e8-4851-8970-48e08ace3ff4 admin admin] PATH_INFO: `/v3/projects/b7c4b94378ee4023a95540195c38b9e2/users/f1646b8caaa343f5adabd84982f4bad9/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:56.097172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e596a-34e8-4851-8970-48e08ace3ff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7c4b94378ee4023a95540195c38b9e2, user_id=f1646b8caaa343f5adabd84982f4bad9, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:56.099897 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e596a-34e8-4851-8970-48e08ace3ff4 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:56.124335 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 918/1836] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:56 2022] PUT /identity/v3/projects/b7c4b94378ee4023a95540195c38b9e2/users/f1646b8caaa343f5adabd84982f4bad9/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:56.135830 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd54e5d-cd1b-4d10-8cc4-844228463c22 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:56.136321 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd54e5d-cd1b-4d10-8cc4-844228463c22 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:56.136601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd54e5d-cd1b-4d10-8cc4-844228463c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:56.170327 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-7dd54e5d-cd1b-4d10-8cc4-844228463c22 None None] MFA Rules not processed for user `f1646b8caaa343f5adabd84982f4bad9`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:56.172084 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-7dd54e5d-cd1b-4d10-8cc4-844228463c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:56.318613 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ec12c-34ec-49be-b3a5-ea55804782d5 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:56.319056 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ec12c-34ec-49be-b3a5-ea55804782d5 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:56.319215 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ec12c-34ec-49be-b3a5-ea55804782d5 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:56.320544 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 919/1837] 10.222.0.18 () {54 vars in 921 bytes} [Tue Aug 23 04:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:27:56.327666 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 919/1838] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:56.330992 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3ec12c-34ec-49be-b3a5-ea55804782d5 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:56.357880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd54e5d-cd1b-4d10-8cc4-844228463c22 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:56.503172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3ec12c-34ec-49be-b3a5-ea55804782d5 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:56.514156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ed77e-d771-4b8f-b166-8a9132df78ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:56.514467 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd54e5d-cd1b-4d10-8cc4-844228463c22 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:56.514821 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ed77e-d771-4b8f-b166-8a9132df78ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:56.515099 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ed77e-d771-4b8f-b166-8a9132df78ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:56.515764 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ed77e-d771-4b8f-b166-8a9132df78ef admin admin] PATH_INFO: `/v3/users/7f33544ca4da4f509d057aea6275992f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:56.518828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32912723-3a8a-490b-8d36-938ad5e7d392 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:56.519448 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32912723-3a8a-490b-8d36-938ad5e7d392 service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:56.519722 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32912723-3a8a-490b-8d36-938ad5e7d392 service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:56.520909 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7ed77e-d771-4b8f-b166-8a9132df78ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f33544ca4da4f509d057aea6275992f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:56.521033 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32912723-3a8a-490b-8d36-938ad5e7d392 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:56.524523 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7ed77e-d771-4b8f-b166-8a9132df78ef admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:56.600653 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:56.617384 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32912723-3a8a-490b-8d36-938ad5e7d392 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:56.619428 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32912723-3a8a-490b-8d36-938ad5e7d392 service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:56.622923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-0b7ed77e-d771-4b8f-b166-8a9132df78ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f33544ca4da4f509d057aea6275992f'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:56.637306 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 920/1839] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:56.638983 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f14db-3f8f-46a1-8b48-599461e08f8b None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:56.639273 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f14db-3f8f-46a1-8b48-599461e08f8b None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:56.639543 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f14db-3f8f-46a1-8b48-599461e08f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:56.651254 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-0b7ed77e-d771-4b8f-b166-8a9132df78ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f33544ca4da4f509d057aea6275992f'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:56.680132 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-513f14db-3f8f-46a1-8b48-599461e08f8b None None] MFA Rules not processed for user `cd7b11f479a54adc8d9b8641859b07a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:56.681355 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-513f14db-3f8f-46a1-8b48-599461e08f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:56.693871 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-0b7ed77e-d771-4b8f-b166-8a9132df78ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f33544ca4da4f509d057aea6275992f'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:56.695501 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-0b7ed77e-d771-4b8f-b166-8a9132df78ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f33544ca4da4f509d057aea6275992f'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:56.704032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-0b7ed77e-d771-4b8f-b166-8a9132df78ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f33544ca4da4f509d057aea6275992f'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:56.728957 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 920/1840] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:27:56 2022] DELETE /identity/v3/users/7f33544ca4da4f509d057aea6275992f => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:56.731879 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ed77e-d771-4b8f-b166-8a9132df78ef admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:56.736096 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-0b7ed77e-d771-4b8f-b166-8a9132df78ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:56.858414 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 921/1841] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Aug 23 04:27:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:56.860169 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038d4a8-8129-4389-b792-c51ea3d48d0c None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:56.860767 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038d4a8-8129-4389-b792-c51ea3d48d0c None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:56.861179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038d4a8-8129-4389-b792-c51ea3d48d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:56.886218 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-5038d4a8-8129-4389-b792-c51ea3d48d0c None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:56.887451 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-5038d4a8-8129-4389-b792-c51ea3d48d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:56.985746 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ed77e-d771-4b8f-b166-8a9132df78ef admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:56.994780 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfed99-a59d-499a-bde6-483bd22c528d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:56.995485 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfed99-a59d-499a-bde6-483bd22c528d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:56.995762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfed99-a59d-499a-bde6-483bd22c528d admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:56.996086 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfed99-a59d-499a-bde6-483bd22c528d admin admin] PATH_INFO: `/v3/projects/c7be68e19cc6491db4ee4a9d8221af20` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:56.997882 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cfed99-a59d-499a-bde6-483bd22c528d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7be68e19cc6491db4ee4a9d8221af20)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:57.001342 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cfed99-a59d-499a-bde6-483bd22c528d admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:57.004604 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:57.071763 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 922/1842] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:57.107316 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-c2cfed99-a59d-499a-bde6-483bd22c528d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7be68e19cc6491db4ee4a9d8221af20', 'request_id': 'req-c2cfed99-a59d-499a-bde6-483bd22c528d'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:57.124277 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-c2cfed99-a59d-499a-bde6-483bd22c528d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7be68e19cc6491db4ee4a9d8221af20', 'request_id': 'req-c2cfed99-a59d-499a-bde6-483bd22c528d'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:57.137664 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-c2cfed99-a59d-499a-bde6-483bd22c528d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7be68e19cc6491db4ee4a9d8221af20', 'request_id': 'req-c2cfed99-a59d-499a-bde6-483bd22c528d'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:57.175129 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-c2cfed99-a59d-499a-bde6-483bd22c528d admin admin] The token cache is being invalidate because project c7be68e19cc6491db4ee4a9d8221af20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:27:57.177783 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-c2cfed99-a59d-499a-bde6-483bd22c528d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:57.179896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-c2cfed99-a59d-499a-bde6-483bd22c528d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:57.187959 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 921/1843] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:27:56 2022] DELETE /identity/v3/projects/c7be68e19cc6491db4ee4a9d8221af20 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:57.193343 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bb949-9cef-4b51-ac69-68934a3d35ba None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:57.193633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bb949-9cef-4b51-ac69-68934a3d35ba None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:57.193910 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bb949-9cef-4b51-ac69-68934a3d35ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:57.236734 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-776bb949-9cef-4b51-ac69-68934a3d35ba None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:57.239008 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfed99-a59d-499a-bde6-483bd22c528d admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:57.240629 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-c2cfed99-a59d-499a-bde6-483bd22c528d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:57.242550 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-776bb949-9cef-4b51-ac69-68934a3d35ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:57.365487 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfed99-a59d-499a-bde6-483bd22c528d admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:57.372637 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e9f60-7d51-440f-8758-56d7c35401df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:57.373281 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e9f60-7d51-440f-8758-56d7c35401df service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:57.373561 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e9f60-7d51-440f-8758-56d7c35401df service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:57.373836 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e9f60-7d51-440f-8758-56d7c35401df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:57.376045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-0f7e9f60-7d51-440f-8758-56d7c35401df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:57.402557 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 923/1844] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:57.404181 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b436f-f95b-4caa-ad69-d736ba7793e8 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:57.404560 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b436f-f95b-4caa-ad69-d736ba7793e8 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:57.405057 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b436f-f95b-4caa-ad69-d736ba7793e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:57.420709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7e9f60-7d51-440f-8758-56d7c35401df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:57.422592 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7e9f60-7d51-440f-8758-56d7c35401df service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:57.450361 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 922/1845] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:27:57.451262 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e9f60-7d51-440f-8758-56d7c35401df service neutron] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:57.452648 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-0f7e9f60-7d51-440f-8758-56d7c35401df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:57.472674 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-3d5b436f-f95b-4caa-ad69-d736ba7793e8 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:57.476272 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-3d5b436f-f95b-4caa-ad69-d736ba7793e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:57.491808 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 924/1846] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:57.493075 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b436f-f95b-4caa-ad69-d736ba7793e8 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:57.503696 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b436f-f95b-4caa-ad69-d736ba7793e8 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:57.509483 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d2e610-a2fe-4d86-b8ff-83407c444271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:57.510271 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d2e610-a2fe-4d86-b8ff-83407c444271 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:57.510400 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d2e610-a2fe-4d86-b8ff-83407c444271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:57.510634 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d2e610-a2fe-4d86-b8ff-83407c444271 admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:57.511291 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d2e610-a2fe-4d86-b8ff-83407c444271 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:27:57.511593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d2e610-a2fe-4d86-b8ff-83407c444271 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:57.513907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d2e610-a2fe-4d86-b8ff-83407c444271 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:57.525069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:57.533837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 925/1847] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:27:57.534651 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d2e610-a2fe-4d86-b8ff-83407c444271 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:57.548271 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d2e610-a2fe-4d86-b8ff-83407c444271 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:57.558483 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda09e03-f2d1-4422-bc2f-145391b9ffcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:57.559572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda09e03-f2d1-4422-bc2f-145391b9ffcd admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:57.560025 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda09e03-f2d1-4422-bc2f-145391b9ffcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:57.560431 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda09e03-f2d1-4422-bc2f-145391b9ffcd admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:57.563452 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda09e03-f2d1-4422-bc2f-145391b9ffcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:57.565737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda09e03-f2d1-4422-bc2f-145391b9ffcd admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:57.586057 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e9f60-7d51-440f-8758-56d7c35401df service neutron] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:57.595566 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f070e2c-0918-4cb0-822a-faa146dc5dc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:57.597056 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f070e2c-0918-4cb0-822a-faa146dc5dc7 service nova] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:57.597467 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f070e2c-0918-4cb0-822a-faa146dc5dc7 service nova] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:57.597835 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f070e2c-0918-4cb0-822a-faa146dc5dc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:57.600259 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-6f070e2c-0918-4cb0-822a-faa146dc5dc7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:57.608326 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 926/1848] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:27:57 2022] POST /identity/v3/projects => generated 367 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:57.613033 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda09e03-f2d1-4422-bc2f-145391b9ffcd admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:57.615623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f070e2c-0918-4cb0-822a-faa146dc5dc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:57.618368 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f070e2c-0918-4cb0-822a-faa146dc5dc7 service nova] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:57.673277 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 923/1849] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:27:57.742024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda09e03-f2d1-4422-bc2f-145391b9ffcd admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:57.747947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a68ab2-30e9-4e31-8918-5c6828ce9d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:57.748753 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a68ab2-30e9-4e31-8918-5c6828ce9d56 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:57.749197 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a68ab2-30e9-4e31-8918-5c6828ce9d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:57.750884 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a68ab2-30e9-4e31-8918-5c6828ce9d56 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:57.751631 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a68ab2-30e9-4e31-8918-5c6828ce9d56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:57.754029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a68ab2-30e9-4e31-8918-5c6828ce9d56 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:57.788752 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 927/1850] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:27:57 2022] POST /identity/v3/users => generated 284 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:57.794202 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f070e2c-0918-4cb0-822a-faa146dc5dc7 service nova] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:57.817875 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f070e2c-0918-4cb0-822a-faa146dc5dc7 service nova] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:57.833283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab83d4-918c-4f01-a36e-d266bc49063d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:57.834592 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab83d4-918c-4f01-a36e-d266bc49063d admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:57.835735 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab83d4-918c-4f01-a36e-d266bc49063d admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:57.836159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab83d4-918c-4f01-a36e-d266bc49063d admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:57.836794 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab83d4-918c-4f01-a36e-d266bc49063d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:57.839597 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab83d4-918c-4f01-a36e-d266bc49063d admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:57.857842 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 924/1851] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:57.862013 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a68ab2-30e9-4e31-8918-5c6828ce9d56 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:57.874579 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a68ab2-30e9-4e31-8918-5c6828ce9d56 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:57.881952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c819264-8859-4b2d-aa04-45d8fd2cfe3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:57.882647 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c819264-8859-4b2d-aa04-45d8fd2cfe3b admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:57.883724 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c819264-8859-4b2d-aa04-45d8fd2cfe3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:57.885290 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c819264-8859-4b2d-aa04-45d8fd2cfe3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:57.885290 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c819264-8859-4b2d-aa04-45d8fd2cfe3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:57.887320 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c819264-8859-4b2d-aa04-45d8fd2cfe3b admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:57.903561 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 928/1852] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:57.904807 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab83d4-918c-4f01-a36e-d266bc49063d admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:57.917223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab83d4-918c-4f01-a36e-d266bc49063d admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:57.923238 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef73da4-7e7f-4ea0-b213-5f96015fc3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:57.923937 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef73da4-7e7f-4ea0-b213-5f96015fc3fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:57.924411 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef73da4-7e7f-4ea0-b213-5f96015fc3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:57.924686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef73da4-7e7f-4ea0-b213-5f96015fc3fe admin admin] PATH_INFO: `/v3/projects/3e77c9949ed9495cb60c3b13ecda5883/users/c14fede51c8b448f842428c4c1197251/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:57.932881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:57.954381 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef73da4-7e7f-4ea0-b213-5f96015fc3fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e77c9949ed9495cb60c3b13ecda5883, user_id=c14fede51c8b448f842428c4c1197251, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:57.959426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef73da4-7e7f-4ea0-b213-5f96015fc3fe admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:57.992016 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 925/1853] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:57 2022] PUT /identity/v3/projects/3e77c9949ed9495cb60c3b13ecda5883/users/c14fede51c8b448f842428c4c1197251/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:27:57.998243 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4afd60b-fe3b-4479-bed2-f4d37206d9d1 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:57.998544 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4afd60b-fe3b-4479-bed2-f4d37206d9d1 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:57.998804 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4afd60b-fe3b-4479-bed2-f4d37206d9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:58.052917 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-b4afd60b-fe3b-4479-bed2-f4d37206d9d1 None None] MFA Rules not processed for user `c14fede51c8b448f842428c4c1197251`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:58.054291 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-b4afd60b-fe3b-4479-bed2-f4d37206d9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:58.183721 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 929/1854] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:27:58.190332 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef73da4-7e7f-4ea0-b213-5f96015fc3fe admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:58.311692 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef73da4-7e7f-4ea0-b213-5f96015fc3fe admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:58.324964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23e8fb-3a90-4a9d-9932-4e356271623a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:58.325994 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23e8fb-3a90-4a9d-9932-4e356271623a admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:58.329309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23e8fb-3a90-4a9d-9932-4e356271623a admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:58.329720 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23e8fb-3a90-4a9d-9932-4e356271623a admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:58.330711 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e23e8fb-3a90-4a9d-9932-4e356271623a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:58.337069 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e23e8fb-3a90-4a9d-9932-4e356271623a admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:58.410902 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 926/1855] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:27:58 2022] POST /identity/v3/projects => generated 369 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:58.417175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4afd60b-fe3b-4479-bed2-f4d37206d9d1 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:58.463947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4afd60b-fe3b-4479-bed2-f4d37206d9d1 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:58.548606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:58.587280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf1399-3e64-48f3-90ee-d28ff0c16e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:58.588155 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf1399-3e64-48f3-90ee-d28ff0c16e04 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:58.588959 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf1399-3e64-48f3-90ee-d28ff0c16e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:58.590714 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf1399-3e64-48f3-90ee-d28ff0c16e04 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:58.592684 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf1399-3e64-48f3-90ee-d28ff0c16e04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:58.600840 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf1399-3e64-48f3-90ee-d28ff0c16e04 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:58.662168 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 930/1856] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:27:58 2022] POST /identity/v3/users => generated 291 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:27:58.666139 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23e8fb-3a90-4a9d-9932-4e356271623a admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:58.683781 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23e8fb-3a90-4a9d-9932-4e356271623a admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:58.710869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d21cf11-361f-4a22-8ce5-b6fac4fef6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:58.714193 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21cf11-361f-4a22-8ce5-b6fac4fef6eb admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:58.715605 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21cf11-361f-4a22-8ce5-b6fac4fef6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:58.716832 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21cf11-361f-4a22-8ce5-b6fac4fef6eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:58.719296 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d21cf11-361f-4a22-8ce5-b6fac4fef6eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:58.723626 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d21cf11-361f-4a22-8ce5-b6fac4fef6eb admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:58.776448 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 927/1857] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:58.779778 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf1399-3e64-48f3-90ee-d28ff0c16e04 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:58.796108 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf1399-3e64-48f3-90ee-d28ff0c16e04 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:58.821412 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09937e3e-ea9f-4408-aafc-3a1a09a86066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:58.822334 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09937e3e-ea9f-4408-aafc-3a1a09a86066 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:58.823006 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09937e3e-ea9f-4408-aafc-3a1a09a86066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:58.823197 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09937e3e-ea9f-4408-aafc-3a1a09a86066 admin admin] PATH_INFO: `/v3/projects/074616a489944ed290f0efae06c658e7/users/683cf7fb436249f78d0046ef545e815e/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:58.850172 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09937e3e-ea9f-4408-aafc-3a1a09a86066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=074616a489944ed290f0efae06c658e7, user_id=683cf7fb436249f78d0046ef545e815e, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:58.854139 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09937e3e-ea9f-4408-aafc-3a1a09a86066 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:58.875137 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 931/1858] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:58 2022] PUT /identity/v3/projects/074616a489944ed290f0efae06c658e7/users/683cf7fb436249f78d0046ef545e815e/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:58.879338 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d21cf11-361f-4a22-8ce5-b6fac4fef6eb admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:58.893139 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d21cf11-361f-4a22-8ce5-b6fac4fef6eb admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:58.974422 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c00ff2-5548-442c-8302-0b0a9585a4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:58.975490 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c00ff2-5548-442c-8302-0b0a9585a4c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:58.976004 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c00ff2-5548-442c-8302-0b0a9585a4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:58.976369 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c00ff2-5548-442c-8302-0b0a9585a4c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:58.977116 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c00ff2-5548-442c-8302-0b0a9585a4c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:58.979132 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c00ff2-5548-442c-8302-0b0a9585a4c4 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:58.994139 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 928/1859] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:27:58.998491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09937e3e-ea9f-4408-aafc-3a1a09a86066 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:59.016698 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09937e3e-ea9f-4408-aafc-3a1a09a86066 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:59.056214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c363cd-2be8-40a9-b5af-c610a95a8aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:59.056928 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c363cd-2be8-40a9-b5af-c610a95a8aa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:59.057216 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c363cd-2be8-40a9-b5af-c610a95a8aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:59.057498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c363cd-2be8-40a9-b5af-c610a95a8aa6 admin admin] PATH_INFO: `/v3/projects/074616a489944ed290f0efae06c658e7/users/683cf7fb436249f78d0046ef545e815e/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:59.065335 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c363cd-2be8-40a9-b5af-c610a95a8aa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=074616a489944ed290f0efae06c658e7, user_id=683cf7fb436249f78d0046ef545e815e, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:59.066527 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c00ff2-5548-442c-8302-0b0a9585a4c4 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:59.068482 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c363cd-2be8-40a9-b5af-c610a95a8aa6 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:59.068850 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-96c00ff2-5548-442c-8302-0b0a9585a4c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:59.086892 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c00ff2-5548-442c-8302-0b0a9585a4c4 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:59.099233 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 932/1860] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:27:58 2022] PUT /identity/v3/projects/074616a489944ed290f0efae06c658e7/users/683cf7fb436249f78d0046ef545e815e/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:59.110456 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f38dd-c34a-49a9-ba41-151383bc0be5 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:59.111005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f38dd-c34a-49a9-ba41-151383bc0be5 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:59.111465 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f38dd-c34a-49a9-ba41-151383bc0be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:59.170879 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-3d7f38dd-c34a-49a9-ba41-151383bc0be5 None None] MFA Rules not processed for user `683cf7fb436249f78d0046ef545e815e`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:27:59.171630 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-3d7f38dd-c34a-49a9-ba41-151383bc0be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:59.182453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:59.195482 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755389c4-f4fc-44ce-85c7-5c6d0586565e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:59.196991 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755389c4-f4fc-44ce-85c7-5c6d0586565e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:59.197279 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755389c4-f4fc-44ce-85c7-5c6d0586565e admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:59.197577 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755389c4-f4fc-44ce-85c7-5c6d0586565e admin admin] PATH_INFO: `/v3/projects/44d5bb381bb444a98655392977ec46a9` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:59.199188 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755389c4-f4fc-44ce-85c7-5c6d0586565e admin admin] RBAC: Authorizing `identity:delete_project(project_id=44d5bb381bb444a98655392977ec46a9)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:59.201343 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755389c4-f4fc-44ce-85c7-5c6d0586565e admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:59.298804 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-755389c4-f4fc-44ce-85c7-5c6d0586565e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44d5bb381bb444a98655392977ec46a9', 'request_id': 'req-755389c4-f4fc-44ce-85c7-5c6d0586565e'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:59.307062 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-755389c4-f4fc-44ce-85c7-5c6d0586565e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44d5bb381bb444a98655392977ec46a9', 'request_id': 'req-755389c4-f4fc-44ce-85c7-5c6d0586565e'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:59.308279 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 933/1861] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:27:59 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:27:59.316959 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-755389c4-f4fc-44ce-85c7-5c6d0586565e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44d5bb381bb444a98655392977ec46a9', 'request_id': 'req-755389c4-f4fc-44ce-85c7-5c6d0586565e'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:59.322787 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f38dd-c34a-49a9-ba41-151383bc0be5 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:59.325013 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-3d7f38dd-c34a-49a9-ba41-151383bc0be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:59.333049 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-755389c4-f4fc-44ce-85c7-5c6d0586565e admin admin] The token cache is being invalidate because project 44d5bb381bb444a98655392977ec46a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:27:59.333839 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-755389c4-f4fc-44ce-85c7-5c6d0586565e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:59.334738 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-755389c4-f4fc-44ce-85c7-5c6d0586565e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:59.339455 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 929/1862] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:27:59 2022] DELETE /identity/v3/projects/44d5bb381bb444a98655392977ec46a9 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:27:59.340851 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755389c4-f4fc-44ce-85c7-5c6d0586565e admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:59.342111 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-755389c4-f4fc-44ce-85c7-5c6d0586565e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:59.406712 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f38dd-c34a-49a9-ba41-151383bc0be5 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:59.423624 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755389c4-f4fc-44ce-85c7-5c6d0586565e admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:27:59.428938 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921152b-99ad-4f43-a440-2eef532d4344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:59.429602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921152b-99ad-4f43-a440-2eef532d4344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:59.429799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921152b-99ad-4f43-a440-2eef532d4344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:59.430083 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921152b-99ad-4f43-a440-2eef532d4344 admin admin] PATH_INFO: `/v3/users/96d3865132b7448c88efdfb7c403c2e9` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:59.431824 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921152b-99ad-4f43-a440-2eef532d4344 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96d3865132b7448c88efdfb7c403c2e9)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:59.433316 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921152b-99ad-4f43-a440-2eef532d4344 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:59.452155 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:27:59.482240 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97be548b-aebc-47c9-a689-4cb79a2bcb67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:27:59.482923 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be548b-aebc-47c9-a689-4cb79a2bcb67 service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:27:59.483252 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be548b-aebc-47c9-a689-4cb79a2bcb67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:27:59.483554 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be548b-aebc-47c9-a689-4cb79a2bcb67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:27:59.485989 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-97be548b-aebc-47c9-a689-4cb79a2bcb67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:59.501882 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d921152b-99ad-4f43-a440-2eef532d4344 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96d3865132b7448c88efdfb7c403c2e9'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:59.512802 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d921152b-99ad-4f43-a440-2eef532d4344 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96d3865132b7448c88efdfb7c403c2e9'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:59.531121 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d921152b-99ad-4f43-a440-2eef532d4344 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96d3865132b7448c88efdfb7c403c2e9'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:59.531996 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d921152b-99ad-4f43-a440-2eef532d4344 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96d3865132b7448c88efdfb7c403c2e9'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:59.542591 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d921152b-99ad-4f43-a440-2eef532d4344 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96d3865132b7448c88efdfb7c403c2e9'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:27:59.553083 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 930/1863] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:27:59 2022] DELETE /identity/v3/users/96d3865132b7448c88efdfb7c403c2e9 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:27:59.583711 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97be548b-aebc-47c9-a689-4cb79a2bcb67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:27:59.585489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97be548b-aebc-47c9-a689-4cb79a2bcb67 service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:27:59.598782 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 934/1864] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:27:59 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:27:59.940770 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921152b-99ad-4f43-a440-2eef532d4344 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:27:59.942815 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-d921152b-99ad-4f43-a440-2eef532d4344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:27:59.953649 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:00.063352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921152b-99ad-4f43-a440-2eef532d4344 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:00.080227 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852a0e4-fa30-43de-8a0a-2ae6e3ab9f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:00.081033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852a0e4-fa30-43de-8a0a-2ae6e3ab9f6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:00.083761 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852a0e4-fa30-43de-8a0a-2ae6e3ab9f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:00.084162 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852a0e4-fa30-43de-8a0a-2ae6e3ab9f6e admin admin] PATH_INFO: `/v3/projects/f1beb00eb94d468ebecc0658145fa514` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:00.087243 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852a0e4-fa30-43de-8a0a-2ae6e3ab9f6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1beb00eb94d468ebecc0658145fa514)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:00.090997 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852a0e4-fa30-43de-8a0a-2ae6e3ab9f6e admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:00.173277 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f852a0e4-fa30-43de-8a0a-2ae6e3ab9f6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1beb00eb94d468ebecc0658145fa514', 'request_id': 'req-f852a0e4-fa30-43de-8a0a-2ae6e3ab9f6e'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:00.188546 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f852a0e4-fa30-43de-8a0a-2ae6e3ab9f6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1beb00eb94d468ebecc0658145fa514', 'request_id': 'req-f852a0e4-fa30-43de-8a0a-2ae6e3ab9f6e'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:00.198166 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f852a0e4-fa30-43de-8a0a-2ae6e3ab9f6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1beb00eb94d468ebecc0658145fa514', 'request_id': 'req-f852a0e4-fa30-43de-8a0a-2ae6e3ab9f6e'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:00.213466 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f852a0e4-fa30-43de-8a0a-2ae6e3ab9f6e admin admin] The token cache is being invalidate because project f1beb00eb94d468ebecc0658145fa514 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:28:00.213764 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f852a0e4-fa30-43de-8a0a-2ae6e3ab9f6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:00.214494 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f852a0e4-fa30-43de-8a0a-2ae6e3ab9f6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:00.221481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 931/1865] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:27:59 2022] DELETE /identity/v3/projects/f1beb00eb94d468ebecc0658145fa514 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:00.223669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce16e00-9807-4213-aca4-d42c62d085e3 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:00.224239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce16e00-9807-4213-aca4-d42c62d085e3 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:00.224528 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce16e00-9807-4213-aca4-d42c62d085e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:00.263902 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-fce16e00-9807-4213-aca4-d42c62d085e3 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:00.265258 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-fce16e00-9807-4213-aca4-d42c62d085e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:00.387485 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 935/1866] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:00.396931 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1177d-396d-4393-bc45-7fae47687b33 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:00.397222 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1177d-396d-4393-bc45-7fae47687b33 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:00.397499 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1177d-396d-4393-bc45-7fae47687b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:00.427243 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-cbd1177d-396d-4393-bc45-7fae47687b33 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:00.429277 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-cbd1177d-396d-4393-bc45-7fae47687b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:00.441414 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 932/1867] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:00.445228 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce16e00-9807-4213-aca4-d42c62d085e3 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:00.456477 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce16e00-9807-4213-aca4-d42c62d085e3 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:00.461701 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633756ba-d6fb-4053-b118-5bd92259c559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:00.462560 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633756ba-d6fb-4053-b118-5bd92259c559 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:00.462662 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633756ba-d6fb-4053-b118-5bd92259c559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:00.462862 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633756ba-d6fb-4053-b118-5bd92259c559 admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:00.463525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633756ba-d6fb-4053-b118-5bd92259c559 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:28:00.463830 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633756ba-d6fb-4053-b118-5bd92259c559 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:00.465690 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633756ba-d6fb-4053-b118-5bd92259c559 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:00.481070 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 936/1868] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:28:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:28:00.484882 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1177d-396d-4393-bc45-7fae47687b33 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:00.508928 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1177d-396d-4393-bc45-7fae47687b33 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:00.526307 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f56ce25-8837-4dd2-a115-0314a4f5b083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:00.527186 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56ce25-8837-4dd2-a115-0314a4f5b083 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:00.527538 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56ce25-8837-4dd2-a115-0314a4f5b083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:00.527929 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56ce25-8837-4dd2-a115-0314a4f5b083 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:00.530288 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f56ce25-8837-4dd2-a115-0314a4f5b083 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:00.533390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f56ce25-8837-4dd2-a115-0314a4f5b083 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:00.687999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 933/1869] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:28:00 2022] POST /identity/v3/projects => generated 381 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:28:00.692353 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633756ba-d6fb-4053-b118-5bd92259c559 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:00.703895 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633756ba-d6fb-4053-b118-5bd92259c559 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:00.799369 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98526b6-ceab-4ae3-b33b-7faf70e1f5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:00.801248 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98526b6-ceab-4ae3-b33b-7faf70e1f5c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:00.801607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98526b6-ceab-4ae3-b33b-7faf70e1f5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:00.802145 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98526b6-ceab-4ae3-b33b-7faf70e1f5c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:00.802974 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98526b6-ceab-4ae3-b33b-7faf70e1f5c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:00.807968 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98526b6-ceab-4ae3-b33b-7faf70e1f5c6 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:00.844504 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 937/1870] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:28:00 2022] POST /identity/v3/users => generated 291 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:28:00.846015 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f56ce25-8837-4dd2-a115-0314a4f5b083 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:00.861911 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f56ce25-8837-4dd2-a115-0314a4f5b083 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:00.868494 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c5d28-b16b-4a4d-a7db-40e06d9548c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:00.869131 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c5d28-b16b-4a4d-a7db-40e06d9548c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:00.869566 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c5d28-b16b-4a4d-a7db-40e06d9548c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:00.869973 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c5d28-b16b-4a4d-a7db-40e06d9548c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:00.870645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c5d28-b16b-4a4d-a7db-40e06d9548c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:00.872321 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c5d28-b16b-4a4d-a7db-40e06d9548c8 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:00.886744 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 934/1871] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:00.887847 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98526b6-ceab-4ae3-b33b-7faf70e1f5c6 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:00.903380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98526b6-ceab-4ae3-b33b-7faf70e1f5c6 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:00.909487 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56a106b-3aae-4ebc-bc0e-69362e6f4164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:00.910149 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a106b-3aae-4ebc-bc0e-69362e6f4164 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:00.910396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a106b-3aae-4ebc-bc0e-69362e6f4164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:00.911184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a106b-3aae-4ebc-bc0e-69362e6f4164 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:00.911873 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56a106b-3aae-4ebc-bc0e-69362e6f4164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:00.914227 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56a106b-3aae-4ebc-bc0e-69362e6f4164 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:00.920120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:00.933530 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 938/1872] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:00.937613 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c5d28-b16b-4a4d-a7db-40e06d9548c8 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:00.950141 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c5d28-b16b-4a4d-a7db-40e06d9548c8 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:00.956341 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6177c699-cdf3-4b4f-a7bc-b86623968724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:00.957759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177c699-cdf3-4b4f-a7bc-b86623968724 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:00.958110 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177c699-cdf3-4b4f-a7bc-b86623968724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:00.958881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177c699-cdf3-4b4f-a7bc-b86623968724 admin admin] PATH_INFO: `/v3/projects/57d3d5a2a0514d7687e794f8a678cff9/users/4eb4a476712c492da2313ef426001e33/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:00.976015 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6177c699-cdf3-4b4f-a7bc-b86623968724 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57d3d5a2a0514d7687e794f8a678cff9, user_id=4eb4a476712c492da2313ef426001e33, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:00.978599 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6177c699-cdf3-4b4f-a7bc-b86623968724 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:00.998507 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 935/1873] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:28:00 2022] PUT /identity/v3/projects/57d3d5a2a0514d7687e794f8a678cff9/users/4eb4a476712c492da2313ef426001e33/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:01.003687 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6849ea6-036a-4dee-9c48-187766ce239d None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:01.004028 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6849ea6-036a-4dee-9c48-187766ce239d None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:01.004322 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6849ea6-036a-4dee-9c48-187766ce239d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:01.023704 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-b6849ea6-036a-4dee-9c48-187766ce239d None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:01.024942 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-b6849ea6-036a-4dee-9c48-187766ce239d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:01.087492 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 939/1874] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:01.097929 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6177c699-cdf3-4b4f-a7bc-b86623968724 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:01.099697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-6177c699-cdf3-4b4f-a7bc-b86623968724 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:01.137223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:01.162595 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6177c699-cdf3-4b4f-a7bc-b86623968724 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:01.166149 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a6cde-a742-46e3-9b95-0ffb2cb06cc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:01.166789 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a6cde-a742-46e3-9b95-0ffb2cb06cc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:01.167091 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a6cde-a742-46e3-9b95-0ffb2cb06cc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:01.167330 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a6cde-a742-46e3-9b95-0ffb2cb06cc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:01.179281 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a6cde-a742-46e3-9b95-0ffb2cb06cc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:01.180782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a6cde-a742-46e3-9b95-0ffb2cb06cc1 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:01.191459 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 936/1875] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:28:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:28:01.435190 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6849ea6-036a-4dee-9c48-187766ce239d None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:01.436660 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-b6849ea6-036a-4dee-9c48-187766ce239d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:01.449364 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6849ea6-036a-4dee-9c48-187766ce239d None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:01.461806 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f8788-e11a-47a9-869a-81cfdc874acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:01.462533 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f8788-e11a-47a9-869a-81cfdc874acc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:01.463566 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f8788-e11a-47a9-869a-81cfdc874acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:01.463566 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f8788-e11a-47a9-869a-81cfdc874acc admin admin] PATH_INFO: `/v3/users/c14fede51c8b448f842428c4c1197251` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:01.464565 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f8788-e11a-47a9-869a-81cfdc874acc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c14fede51c8b448f842428c4c1197251)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:01.466575 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f8788-e11a-47a9-869a-81cfdc874acc admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:01.520828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-bf4f8788-e11a-47a9-869a-81cfdc874acc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c14fede51c8b448f842428c4c1197251'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:01.522031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-bf4f8788-e11a-47a9-869a-81cfdc874acc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c14fede51c8b448f842428c4c1197251'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:01.532633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-bf4f8788-e11a-47a9-869a-81cfdc874acc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c14fede51c8b448f842428c4c1197251'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:01.555318 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-bf4f8788-e11a-47a9-869a-81cfdc874acc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c14fede51c8b448f842428c4c1197251'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:01.565434 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-bf4f8788-e11a-47a9-869a-81cfdc874acc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c14fede51c8b448f842428c4c1197251'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:01.575981 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 940/1876] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:28:01 2022] DELETE /identity/v3/users/c14fede51c8b448f842428c4c1197251 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:01.581263 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9382d-d7bd-4fe1-b610-1b405680735f None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:01.581852 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9382d-d7bd-4fe1-b610-1b405680735f None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:01.581937 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9382d-d7bd-4fe1-b610-1b405680735f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:01.607828 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-3fd9382d-d7bd-4fe1-b610-1b405680735f None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:01.609911 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-3fd9382d-d7bd-4fe1-b610-1b405680735f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:01.692287 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 937/1877] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:01.721305 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f8788-e11a-47a9-869a-81cfdc874acc admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:01.723788 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-bf4f8788-e11a-47a9-869a-81cfdc874acc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:01.845230 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f8788-e11a-47a9-869a-81cfdc874acc admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:01.850221 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bad704-a875-4a68-80ab-c0709634fcce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:01.850742 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bad704-a875-4a68-80ab-c0709634fcce service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:01.851512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bad704-a875-4a68-80ab-c0709634fcce service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:01.851892 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bad704-a875-4a68-80ab-c0709634fcce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:01.866793 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bad704-a875-4a68-80ab-c0709634fcce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:01.868308 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bad704-a875-4a68-80ab-c0709634fcce service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:01.887667 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 941/1878] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:28:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:28:01.895723 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9382d-d7bd-4fe1-b610-1b405680735f None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:01.897552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-3fd9382d-d7bd-4fe1-b610-1b405680735f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:01.977973 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9382d-d7bd-4fe1-b610-1b405680735f None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:01.981907 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ae7c4-ec93-4d30-8c46-fe4218e14d81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:01.983064 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ae7c4-ec93-4d30-8c46-fe4218e14d81 service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:01.983355 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ae7c4-ec93-4d30-8c46-fe4218e14d81 service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:01.983777 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ae7c4-ec93-4d30-8c46-fe4218e14d81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:01.985724 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-f37ae7c4-ec93-4d30-8c46-fe4218e14d81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:02.060463 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37ae7c4-ec93-4d30-8c46-fe4218e14d81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:02.061929 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37ae7c4-ec93-4d30-8c46-fe4218e14d81 service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:02.073127 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 938/1879] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:28:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:28:02.563720 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b932c44-556f-48e3-9f81-3cb2389a2d57 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:02.564233 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b932c44-556f-48e3-9f81-3cb2389a2d57 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:02.564582 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b932c44-556f-48e3-9f81-3cb2389a2d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:02.569648 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ae7c4-ec93-4d30-8c46-fe4218e14d81 service cinder] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:02.571256 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-f37ae7c4-ec93-4d30-8c46-fe4218e14d81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:02.588215 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ae7c4-ec93-4d30-8c46-fe4218e14d81 service cinder] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:02.590249 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:02.595640 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e4f74-09dc-46d7-8cc6-4663948b0aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:02.597104 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e4f74-09dc-46d7-8cc6-4663948b0aa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:02.597698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e4f74-09dc-46d7-8cc6-4663948b0aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:02.598255 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e4f74-09dc-46d7-8cc6-4663948b0aa9 admin admin] PATH_INFO: `/v3/projects/3e77c9949ed9495cb60c3b13ecda5883` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:02.599580 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-7b932c44-556f-48e3-9f81-3cb2389a2d57 None None] MFA Rules not processed for user `4eb4a476712c492da2313ef426001e33`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:02.599775 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e4f74-09dc-46d7-8cc6-4663948b0aa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e77c9949ed9495cb60c3b13ecda5883)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:02.602056 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e4f74-09dc-46d7-8cc6-4663948b0aa9 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:02.602791 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-7b932c44-556f-48e3-9f81-3cb2389a2d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:02.626615 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:02.653509 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f13e4f74-09dc-46d7-8cc6-4663948b0aa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e77c9949ed9495cb60c3b13ecda5883', 'request_id': 'req-f13e4f74-09dc-46d7-8cc6-4663948b0aa9'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:02.661264 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f13e4f74-09dc-46d7-8cc6-4663948b0aa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e77c9949ed9495cb60c3b13ecda5883', 'request_id': 'req-f13e4f74-09dc-46d7-8cc6-4663948b0aa9'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:02.669758 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f13e4f74-09dc-46d7-8cc6-4663948b0aa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e77c9949ed9495cb60c3b13ecda5883', 'request_id': 'req-f13e4f74-09dc-46d7-8cc6-4663948b0aa9'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:02.684170 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f13e4f74-09dc-46d7-8cc6-4663948b0aa9 admin admin] The token cache is being invalidate because project 3e77c9949ed9495cb60c3b13ecda5883 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:28:02.684452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f13e4f74-09dc-46d7-8cc6-4663948b0aa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:02.685423 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f13e4f74-09dc-46d7-8cc6-4663948b0aa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:02.686903 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 939/1880] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:28:02 2022] DELETE /identity/v3/projects/3e77c9949ed9495cb60c3b13ecda5883 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:02.691064 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e4f74-09dc-46d7-8cc6-4663948b0aa9 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:02.692444 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-f13e4f74-09dc-46d7-8cc6-4663948b0aa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:02.692597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 942/1881] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:02.702457 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b932c44-556f-48e3-9f81-3cb2389a2d57 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:02.712663 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b932c44-556f-48e3-9f81-3cb2389a2d57 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:02.783692 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e4f74-09dc-46d7-8cc6-4663948b0aa9 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:02.790665 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b54e1-d384-4519-9b24-39500e00d931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:02.791601 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b54e1-d384-4519-9b24-39500e00d931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:02.792202 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b54e1-d384-4519-9b24-39500e00d931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:02.792692 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b54e1-d384-4519-9b24-39500e00d931 admin admin] PATH_INFO: `/v3/users/683cf7fb436249f78d0046ef545e815e` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:02.794505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740b54e1-d384-4519-9b24-39500e00d931 admin admin] RBAC: Authorizing `identity:delete_user(user_id=683cf7fb436249f78d0046ef545e815e)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:02.794770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3ee7d-fc4a-4c02-ab67-c2094c4c0b6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:02.795391 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3ee7d-fc4a-4c02-ab67-c2094c4c0b6b service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:02.795666 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3ee7d-fc4a-4c02-ab67-c2094c4c0b6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:02.795957 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3ee7d-fc4a-4c02-ab67-c2094c4c0b6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:02.797872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-15c3ee7d-fc4a-4c02-ab67-c2094c4c0b6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:02.798257 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740b54e1-d384-4519-9b24-39500e00d931 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:02.857813 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-740b54e1-d384-4519-9b24-39500e00d931 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '683cf7fb436249f78d0046ef545e815e'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:02.866358 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-740b54e1-d384-4519-9b24-39500e00d931 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '683cf7fb436249f78d0046ef545e815e'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:02.877284 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c3ee7d-fc4a-4c02-ab67-c2094c4c0b6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:02.878967 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c3ee7d-fc4a-4c02-ab67-c2094c4c0b6b service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:02.882067 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-740b54e1-d384-4519-9b24-39500e00d931 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '683cf7fb436249f78d0046ef545e815e'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:02.882930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-740b54e1-d384-4519-9b24-39500e00d931 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '683cf7fb436249f78d0046ef545e815e'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:02.891343 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-740b54e1-d384-4519-9b24-39500e00d931 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '683cf7fb436249f78d0046ef545e815e'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:02.892617 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 943/1882] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:28:02 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:28:02.898089 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 940/1883] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:28:02 2022] DELETE /identity/v3/users/683cf7fb436249f78d0046ef545e815e => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:28:03.356708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3ee7d-fc4a-4c02-ab67-c2094c4c0b6b service cinder] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:03.368979 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3ee7d-fc4a-4c02-ab67-c2094c4c0b6b service cinder] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:03.447058 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:03.456473 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40dd4b-e131-4c1e-91de-b692cf961c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:03.457122 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40dd4b-e131-4c1e-91de-b692cf961c2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:03.457401 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40dd4b-e131-4c1e-91de-b692cf961c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:03.458090 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40dd4b-e131-4c1e-91de-b692cf961c2b admin admin] PATH_INFO: `/v3/projects/074616a489944ed290f0efae06c658e7` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:03.459357 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40dd4b-e131-4c1e-91de-b692cf961c2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=074616a489944ed290f0efae06c658e7)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:03.461281 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40dd4b-e131-4c1e-91de-b692cf961c2b admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:03.512413 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-7c40dd4b-e131-4c1e-91de-b692cf961c2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '074616a489944ed290f0efae06c658e7', 'request_id': 'req-7c40dd4b-e131-4c1e-91de-b692cf961c2b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:03.519703 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-7c40dd4b-e131-4c1e-91de-b692cf961c2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '074616a489944ed290f0efae06c658e7', 'request_id': 'req-7c40dd4b-e131-4c1e-91de-b692cf961c2b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:03.528879 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-7c40dd4b-e131-4c1e-91de-b692cf961c2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '074616a489944ed290f0efae06c658e7', 'request_id': 'req-7c40dd4b-e131-4c1e-91de-b692cf961c2b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:03.541749 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-7c40dd4b-e131-4c1e-91de-b692cf961c2b admin admin] The token cache is being invalidate because project 074616a489944ed290f0efae06c658e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:28:03.542288 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-7c40dd4b-e131-4c1e-91de-b692cf961c2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:03.542933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-7c40dd4b-e131-4c1e-91de-b692cf961c2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:03.544773 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 944/1884] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:28:03 2022] DELETE /identity/v3/projects/074616a489944ed290f0efae06c658e7 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:03.552971 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71c21b-418b-4c68-a829-43e1a9651438 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:03.553266 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71c21b-418b-4c68-a829-43e1a9651438 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:03.553537 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71c21b-418b-4c68-a829-43e1a9651438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:03.583132 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-de71c21b-418b-4c68-a829-43e1a9651438 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:03.584400 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-de71c21b-418b-4c68-a829-43e1a9651438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:03.605523 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:03.648401 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 941/1885] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:03.657735 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fa4c1-c1b4-402a-aa89-ae822f479cc9 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:03.658229 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fa4c1-c1b4-402a-aa89-ae822f479cc9 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:03.658352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fa4c1-c1b4-402a-aa89-ae822f479cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:03.681580 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-466fa4c1-c1b4-402a-aa89-ae822f479cc9 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:03.682840 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-466fa4c1-c1b4-402a-aa89-ae822f479cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:03.692245 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 945/1886] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:03.697294 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71c21b-418b-4c68-a829-43e1a9651438 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:03.708770 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71c21b-418b-4c68-a829-43e1a9651438 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:03.714363 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f725b-f10e-4a30-b0f6-d64621a017cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:03.714991 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f725b-f10e-4a30-b0f6-d64621a017cb admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:03.715312 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f725b-f10e-4a30-b0f6-d64621a017cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:03.715549 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f725b-f10e-4a30-b0f6-d64621a017cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:03.716249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f725b-f10e-4a30-b0f6-d64621a017cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:28:03.716554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f725b-f10e-4a30-b0f6-d64621a017cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:03.718171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f725b-f10e-4a30-b0f6-d64621a017cb admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:03.731412 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 942/1887] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:28:03.736114 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466fa4c1-c1b4-402a-aa89-ae822f479cc9 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:03.745740 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466fa4c1-c1b4-402a-aa89-ae822f479cc9 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:03.751375 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486d05f-7290-4e7c-ace3-73f13048d1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:03.751905 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486d05f-7290-4e7c-ace3-73f13048d1c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:03.752228 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486d05f-7290-4e7c-ace3-73f13048d1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:03.752497 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486d05f-7290-4e7c-ace3-73f13048d1c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:03.753122 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d486d05f-7290-4e7c-ace3-73f13048d1c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:03.755037 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d486d05f-7290-4e7c-ace3-73f13048d1c3 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:03.786212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 946/1888] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:28:03 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:28:03.792687 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f725b-f10e-4a30-b0f6-d64621a017cb admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:03.803429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f725b-f10e-4a30-b0f6-d64621a017cb admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:03.864041 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a670c95-8285-4b10-9b96-15de8a4d53a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:03.864662 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a670c95-8285-4b10-9b96-15de8a4d53a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:03.864938 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a670c95-8285-4b10-9b96-15de8a4d53a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:03.865208 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a670c95-8285-4b10-9b96-15de8a4d53a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:03.865865 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a670c95-8285-4b10-9b96-15de8a4d53a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:03.867818 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a670c95-8285-4b10-9b96-15de8a4d53a2 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:03.892740 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 943/1889] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:28:03 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:28:03.897015 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486d05f-7290-4e7c-ace3-73f13048d1c3 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:03.907974 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486d05f-7290-4e7c-ace3-73f13048d1c3 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:03.913631 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb96ad-979c-40c1-a61e-f7e534ff4f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:03.914229 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb96ad-979c-40c1-a61e-f7e534ff4f39 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:03.914503 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb96ad-979c-40c1-a61e-f7e534ff4f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:03.914766 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb96ad-979c-40c1-a61e-f7e534ff4f39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:03.915404 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb96ad-979c-40c1-a61e-f7e534ff4f39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:03.917049 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb96ad-979c-40c1-a61e-f7e534ff4f39 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:03.927384 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 947/1890] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:28:03.931370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a670c95-8285-4b10-9b96-15de8a4d53a2 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:03.941601 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a670c95-8285-4b10-9b96-15de8a4d53a2 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:03.946497 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f38a8-7382-4b47-a980-203f5cd205f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:03.947100 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f38a8-7382-4b47-a980-203f5cd205f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:03.947400 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f38a8-7382-4b47-a980-203f5cd205f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:03.947708 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f38a8-7382-4b47-a980-203f5cd205f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:03.948355 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f38a8-7382-4b47-a980-203f5cd205f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:03.949941 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f38a8-7382-4b47-a980-203f5cd205f4 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:03.963361 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 944/1891] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:03.968204 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb96ad-979c-40c1-a61e-f7e534ff4f39 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:03.984369 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb96ad-979c-40c1-a61e-f7e534ff4f39 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:03.990518 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94aa9ff-124d-4bdc-9cf0-4068725c903f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:03.991736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94aa9ff-124d-4bdc-9cf0-4068725c903f admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:03.991736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94aa9ff-124d-4bdc-9cf0-4068725c903f admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:03.991736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94aa9ff-124d-4bdc-9cf0-4068725c903f admin admin] PATH_INFO: `/v3/projects/df7627d189c348a7aa383fca5a61e5d1/users/0edc88b6b3944ceaab0a5cc563c1fdb9/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:04.007293 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94aa9ff-124d-4bdc-9cf0-4068725c903f admin admin] RBAC: Authorizing `identity:create_grant(project_id=df7627d189c348a7aa383fca5a61e5d1, user_id=0edc88b6b3944ceaab0a5cc563c1fdb9, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:04.009886 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94aa9ff-124d-4bdc-9cf0-4068725c903f admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:04.034141 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 948/1892] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:28:03 2022] PUT /identity/v3/projects/df7627d189c348a7aa383fca5a61e5d1/users/0edc88b6b3944ceaab0a5cc563c1fdb9/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:04.035804 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46a89d1-af98-4478-b59a-cf5d560329bd None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:04.036129 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46a89d1-af98-4478-b59a-cf5d560329bd None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:04.036406 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46a89d1-af98-4478-b59a-cf5d560329bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:04.068634 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-f46a89d1-af98-4478-b59a-cf5d560329bd None None] MFA Rules not processed for user `0edc88b6b3944ceaab0a5cc563c1fdb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:04.069989 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-f46a89d1-af98-4478-b59a-cf5d560329bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:04.137505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 945/1893] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:04.142203 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94aa9ff-124d-4bdc-9cf0-4068725c903f admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:04.205250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94aa9ff-124d-4bdc-9cf0-4068725c903f admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:04.209958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c43c9-95ca-4372-877a-1455f359b5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:04.210575 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c43c9-95ca-4372-877a-1455f359b5d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:04.210858 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c43c9-95ca-4372-877a-1455f359b5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:04.211133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c43c9-95ca-4372-877a-1455f359b5d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:04.212707 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5c43c9-95ca-4372-877a-1455f359b5d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:04.214117 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5c43c9-95ca-4372-877a-1455f359b5d0 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:04.258600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 949/1894] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:28:04 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:28:04.264399 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46a89d1-af98-4478-b59a-cf5d560329bd None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:04.273818 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46a89d1-af98-4478-b59a-cf5d560329bd None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:04.293527 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:04.334932 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77580c43-2308-49ff-acb5-c25d168eb031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:04.335595 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77580c43-2308-49ff-acb5-c25d168eb031 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:04.335863 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77580c43-2308-49ff-acb5-c25d168eb031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:04.336168 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77580c43-2308-49ff-acb5-c25d168eb031 admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:04.336828 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77580c43-2308-49ff-acb5-c25d168eb031 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:04.338775 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77580c43-2308-49ff-acb5-c25d168eb031 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:04.362184 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 946/1895] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:28:04 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:28:04.366185 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c43c9-95ca-4372-877a-1455f359b5d0 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:04.376780 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c43c9-95ca-4372-877a-1455f359b5d0 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:04.381522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e1451-a221-4714-b119-7fafe6ecd594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:04.382128 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e1451-a221-4714-b119-7fafe6ecd594 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:04.382406 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e1451-a221-4714-b119-7fafe6ecd594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:04.382676 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e1451-a221-4714-b119-7fafe6ecd594 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:04.383326 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e1451-a221-4714-b119-7fafe6ecd594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:04.384954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e1451-a221-4714-b119-7fafe6ecd594 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:04.386324 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:04.397821 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 950/1896] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:04.403652 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77580c43-2308-49ff-acb5-c25d168eb031 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:04.425808 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77580c43-2308-49ff-acb5-c25d168eb031 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:04.433016 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c313fca-c52b-4e70-b595-1386be3089b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:04.433703 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c313fca-c52b-4e70-b595-1386be3089b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:04.434036 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c313fca-c52b-4e70-b595-1386be3089b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:04.434311 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c313fca-c52b-4e70-b595-1386be3089b9 admin admin] PATH_INFO: `/v3/projects/cef43e27688e44d1bcc6967cf7d33321/users/e9a77c678ee44d7f8cdce1226915682b/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:04.471323 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c313fca-c52b-4e70-b595-1386be3089b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cef43e27688e44d1bcc6967cf7d33321, user_id=e9a77c678ee44d7f8cdce1226915682b, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:04.474568 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c313fca-c52b-4e70-b595-1386be3089b9 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:04.494875 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 947/1897] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:28:04 2022] PUT /identity/v3/projects/cef43e27688e44d1bcc6967cf7d33321/users/e9a77c678ee44d7f8cdce1226915682b/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:04.499171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e1451-a221-4714-b119-7fafe6ecd594 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:04.509795 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e1451-a221-4714-b119-7fafe6ecd594 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:04.579571 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60857d2c-2324-4b90-9bf0-9516b1a28c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:04.580704 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60857d2c-2324-4b90-9bf0-9516b1a28c37 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:04.580991 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60857d2c-2324-4b90-9bf0-9516b1a28c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:04.581322 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60857d2c-2324-4b90-9bf0-9516b1a28c37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:04.582195 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60857d2c-2324-4b90-9bf0-9516b1a28c37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:04.583984 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60857d2c-2324-4b90-9bf0-9516b1a28c37 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:04.596232 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 951/1898] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:04.600874 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c313fca-c52b-4e70-b595-1386be3089b9 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:04.611519 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c313fca-c52b-4e70-b595-1386be3089b9 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:04.616198 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95892b3d-b8ac-4b0c-911e-27e5b7344c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:04.616871 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95892b3d-b8ac-4b0c-911e-27e5b7344c5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:04.617153 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95892b3d-b8ac-4b0c-911e-27e5b7344c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:04.617426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95892b3d-b8ac-4b0c-911e-27e5b7344c5e admin admin] PATH_INFO: `/v3/projects/cef43e27688e44d1bcc6967cf7d33321/users/e9a77c678ee44d7f8cdce1226915682b/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:04.619316 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95892b3d-b8ac-4b0c-911e-27e5b7344c5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cef43e27688e44d1bcc6967cf7d33321, user_id=e9a77c678ee44d7f8cdce1226915682b, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:04.622168 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95892b3d-b8ac-4b0c-911e-27e5b7344c5e admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:04.642091 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 948/1899] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:28:04 2022] PUT /identity/v3/projects/cef43e27688e44d1bcc6967cf7d33321/users/e9a77c678ee44d7f8cdce1226915682b/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:04.645544 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17021547-c897-4a23-97af-4c9fc2b1e030 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:04.645901 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17021547-c897-4a23-97af-4c9fc2b1e030 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:04.646104 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17021547-c897-4a23-97af-4c9fc2b1e030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:04.676034 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-17021547-c897-4a23-97af-4c9fc2b1e030 None None] MFA Rules not processed for user `e9a77c678ee44d7f8cdce1226915682b`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:04.677247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-17021547-c897-4a23-97af-4c9fc2b1e030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:04.749839 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 952/1900] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:04 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:04.762757 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95892b3d-b8ac-4b0c-911e-27e5b7344c5e admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:04.764549 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-95892b3d-b8ac-4b0c-911e-27e5b7344c5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:04.823219 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95892b3d-b8ac-4b0c-911e-27e5b7344c5e admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:04.826971 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287311e-f193-43d0-8d58-120374c9b698 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:04.827584 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287311e-f193-43d0-8d58-120374c9b698 service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:04.828019 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287311e-f193-43d0-8d58-120374c9b698 service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:04.828325 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287311e-f193-43d0-8d58-120374c9b698 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:04.837481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1287311e-f193-43d0-8d58-120374c9b698 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:04.838925 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1287311e-f193-43d0-8d58-120374c9b698 service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:04.849201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 949/1901] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:28:04 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:28:05.215093 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17021547-c897-4a23-97af-4c9fc2b1e030 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:05.225126 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17021547-c897-4a23-97af-4c9fc2b1e030 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:05.230721 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ec855-86aa-483b-902e-ce1e516f0265 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9a77c678ee44d7f8cdce1226915682b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cef43e27688e44d1bcc6967cf7d33321', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:05.231320 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ec855-86aa-483b-902e-ce1e516f0265 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:05.231596 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ec855-86aa-483b-902e-ce1e516f0265 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:05.231864 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ec855-86aa-483b-902e-ce1e516f0265 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:05.232544 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ec855-86aa-483b-902e-ce1e516f0265 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:28:05.232850 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ec855-86aa-483b-902e-ce1e516f0265 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:05.234545 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ec855-86aa-483b-902e-ce1e516f0265 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:05.247660 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 953/1902] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:28:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:28:05.252791 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287311e-f193-43d0-8d58-120374c9b698 service cinder] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:05.263119 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287311e-f193-43d0-8d58-120374c9b698 service cinder] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:05.269248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0ba14-86cc-4c3f-a4e1-1f18a674cd5b tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9a77c678ee44d7f8cdce1226915682b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cef43e27688e44d1bcc6967cf7d33321', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:05.269951 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0ba14-86cc-4c3f-a4e1-1f18a674cd5b tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:05.270166 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0ba14-86cc-4c3f-a4e1-1f18a674cd5b tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:05.270436 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0ba14-86cc-4c3f-a4e1-1f18a674cd5b tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:05.271060 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c0ba14-86cc-4c3f-a4e1-1f18a674cd5b tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:05.273030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c0ba14-86cc-4c3f-a4e1-1f18a674cd5b tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:05.309547 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 950/1903] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:28:05 2022] POST /identity/v3/projects => generated 353 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:28:05.311123 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ec855-86aa-483b-902e-ce1e516f0265 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:05.331257 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ec855-86aa-483b-902e-ce1e516f0265 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:05.405466 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5fc17-175f-4fd7-a1bc-00779c1fbcd6 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9a77c678ee44d7f8cdce1226915682b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cef43e27688e44d1bcc6967cf7d33321', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:05.406135 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5fc17-175f-4fd7-a1bc-00779c1fbcd6 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:05.406415 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5fc17-175f-4fd7-a1bc-00779c1fbcd6 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:05.406686 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5fc17-175f-4fd7-a1bc-00779c1fbcd6 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:05.407334 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5fc17-175f-4fd7-a1bc-00779c1fbcd6 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:28:05.407639 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5fc17-175f-4fd7-a1bc-00779c1fbcd6 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:05.409451 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5fc17-175f-4fd7-a1bc-00779c1fbcd6 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:05.417308 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:05.424265 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 954/1904] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:28:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:28:05.462738 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0ba14-86cc-4c3f-a4e1-1f18a674cd5b tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:05.474505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0ba14-86cc-4c3f-a4e1-1f18a674cd5b tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:05.479563 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3c159-094d-4b96-9f97-4c08ab2ce622 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9a77c678ee44d7f8cdce1226915682b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cef43e27688e44d1bcc6967cf7d33321', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:05.480353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3c159-094d-4b96-9f97-4c08ab2ce622 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:05.480552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3c159-094d-4b96-9f97-4c08ab2ce622 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:05.480828 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3c159-094d-4b96-9f97-4c08ab2ce622 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] PATH_INFO: `/v3/projects/882c9e7991ad4149866e29742318cb67` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:05.483161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af3c159-094d-4b96-9f97-4c08ab2ce622 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] RBAC: Authorizing `identity:delete_project(project_id=882c9e7991ad4149866e29742318cb67)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:05.485355 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af3c159-094d-4b96-9f97-4c08ab2ce622 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:05.574721 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e4f83-0c79-4b78-93bb-f95108ed114d None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:05.575029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e4f83-0c79-4b78-93bb-f95108ed114d None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:05.575541 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e4f83-0c79-4b78-93bb-f95108ed114d None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:05.577323 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 955/1905] 10.222.0.18 () {54 vars in 921 bytes} [Tue Aug 23 04:28:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:28:05.577840 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-5af3c159-094d-4b96-9f97-4c08ab2ce622 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '882c9e7991ad4149866e29742318cb67', 'request_id': 'req-5af3c159-094d-4b96-9f97-4c08ab2ce622'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:05.579519 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=90,keystone:INSERT=6,keystone:DELETE=4 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:05.587415 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-5af3c159-094d-4b96-9f97-4c08ab2ce622 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '882c9e7991ad4149866e29742318cb67', 'request_id': 'req-5af3c159-094d-4b96-9f97-4c08ab2ce622'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:05.597698 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810215a1-3aa1-45a3-a5a8-e95263f87be6 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:05.598128 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810215a1-3aa1-45a3-a5a8-e95263f87be6 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:05.598506 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810215a1-3aa1-45a3-a5a8-e95263f87be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:05.598881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-5af3c159-094d-4b96-9f97-4c08ab2ce622 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '882c9e7991ad4149866e29742318cb67', 'request_id': 'req-5af3c159-094d-4b96-9f97-4c08ab2ce622'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:05.617868 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-5af3c159-094d-4b96-9f97-4c08ab2ce622 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] The token cache is being invalidate because project 882c9e7991ad4149866e29742318cb67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:28:05.617868 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-5af3c159-094d-4b96-9f97-4c08ab2ce622 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:05.619247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-5af3c159-094d-4b96-9f97-4c08ab2ce622 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:05.625782 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-810215a1-3aa1-45a3-a5a8-e95263f87be6 None None] MFA Rules not processed for user `cd7b11f479a54adc8d9b8641859b07a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:05.627587 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-810215a1-3aa1-45a3-a5a8-e95263f87be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:05.636150 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 951/1906] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:28:05 2022] DELETE /identity/v3/projects/882c9e7991ad4149866e29742318cb67 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:05.722158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 956/1907] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Aug 23 04:28:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:05.960736 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3c159-094d-4b96-9f97-4c08ab2ce622 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:05.962897 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-5af3c159-094d-4b96-9f97-4c08ab2ce622 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:05.985918 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810215a1-3aa1-45a3-a5a8-e95263f87be6 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:05.988377 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-810215a1-3aa1-45a3-a5a8-e95263f87be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:06.004587 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810215a1-3aa1-45a3-a5a8-e95263f87be6 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:06.013638 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c19e8-a1b4-4c18-a843-b27655a1eb5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:06.014645 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c19e8-a1b4-4c18-a843-b27655a1eb5e service nova] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:06.015186 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c19e8-a1b4-4c18-a843-b27655a1eb5e service nova] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:06.016050 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c19e8-a1b4-4c18-a843-b27655a1eb5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:06.026329 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c19e8-a1b4-4c18-a843-b27655a1eb5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:06.028016 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c19e8-a1b4-4c18-a843-b27655a1eb5e service nova] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:06.039046 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 957/1908] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:28:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:28:06.046466 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3c159-094d-4b96-9f97-4c08ab2ce622 tempest-VolumeQuotaClassesTest-829906278 tempest-VolumeQuotaClassesTest-829906278-project-admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:06.051105 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3f054-60ad-4714-bea5-2f96f8e15d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:06.051745 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3f054-60ad-4714-bea5-2f96f8e15d3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:06.052056 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3f054-60ad-4714-bea5-2f96f8e15d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:06.052328 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3f054-60ad-4714-bea5-2f96f8e15d3d admin admin] PATH_INFO: `/v3/users/0edc88b6b3944ceaab0a5cc563c1fdb9` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:06.053557 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f3f054-60ad-4714-bea5-2f96f8e15d3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0edc88b6b3944ceaab0a5cc563c1fdb9)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:06.055494 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f3f054-60ad-4714-bea5-2f96f8e15d3d admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:06.117802 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-04f3f054-60ad-4714-bea5-2f96f8e15d3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0edc88b6b3944ceaab0a5cc563c1fdb9'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:06.125414 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-04f3f054-60ad-4714-bea5-2f96f8e15d3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0edc88b6b3944ceaab0a5cc563c1fdb9'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:06.138190 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-04f3f054-60ad-4714-bea5-2f96f8e15d3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0edc88b6b3944ceaab0a5cc563c1fdb9'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:06.138878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-04f3f054-60ad-4714-bea5-2f96f8e15d3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0edc88b6b3944ceaab0a5cc563c1fdb9'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:06.147289 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-04f3f054-60ad-4714-bea5-2f96f8e15d3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0edc88b6b3944ceaab0a5cc563c1fdb9'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:06.154393 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 952/1909] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:28:05 2022] DELETE /identity/v3/users/0edc88b6b3944ceaab0a5cc563c1fdb9 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:06.158581 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2e65d-9681-4eb3-b53f-c698114b398c None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:06.158817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2e65d-9681-4eb3-b53f-c698114b398c None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:06.159107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2e65d-9681-4eb3-b53f-c698114b398c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:06.178433 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-12a2e65d-9681-4eb3-b53f-c698114b398c None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:06.179613 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-12a2e65d-9681-4eb3-b53f-c698114b398c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:06.241480 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 958/1910] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:06.254405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3f054-60ad-4714-bea5-2f96f8e15d3d admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:06.256800 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-04f3f054-60ad-4714-bea5-2f96f8e15d3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:06.326336 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3f054-60ad-4714-bea5-2f96f8e15d3d admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:06.332871 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77afba1-f9c2-457e-8c0d-526db59d5b44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:06.333719 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77afba1-f9c2-457e-8c0d-526db59d5b44 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:06.334051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77afba1-f9c2-457e-8c0d-526db59d5b44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:06.334452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77afba1-f9c2-457e-8c0d-526db59d5b44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:06.348274 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77afba1-f9c2-457e-8c0d-526db59d5b44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:06.349642 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77afba1-f9c2-457e-8c0d-526db59d5b44 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:06.354589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:06.360055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 953/1911] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:28:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:28:06.426050 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24550071-4e49-4c4e-9580-5aaf607e5733 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:06.426413 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24550071-4e49-4c4e-9580-5aaf607e5733 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:06.426814 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24550071-4e49-4c4e-9580-5aaf607e5733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:06.447737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-24550071-4e49-4c4e-9580-5aaf607e5733 None None] MFA Rules not processed for user `4eb4a476712c492da2313ef426001e33`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:06.449284 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-24550071-4e49-4c4e-9580-5aaf607e5733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:06.516276 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 959/1912] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:07.857881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77afba1-f9c2-457e-8c0d-526db59d5b44 service neutron] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:07.860310 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-e77afba1-f9c2-457e-8c0d-526db59d5b44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:07.879255 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77afba1-f9c2-457e-8c0d-526db59d5b44 service neutron] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:07.884859 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea1fcc-038c-4e36-ac26-e3bcb785748b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:07.885475 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea1fcc-038c-4e36-ac26-e3bcb785748b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:07.885937 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea1fcc-038c-4e36-ac26-e3bcb785748b admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:07.886378 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea1fcc-038c-4e36-ac26-e3bcb785748b admin admin] PATH_INFO: `/v3/projects/df7627d189c348a7aa383fca5a61e5d1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:07.887730 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea1fcc-038c-4e36-ac26-e3bcb785748b admin admin] RBAC: Authorizing `identity:delete_project(project_id=df7627d189c348a7aa383fca5a61e5d1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:07.889686 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea1fcc-038c-4e36-ac26-e3bcb785748b admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:07.966757 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-99ea1fcc-038c-4e36-ac26-e3bcb785748b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df7627d189c348a7aa383fca5a61e5d1', 'request_id': 'req-99ea1fcc-038c-4e36-ac26-e3bcb785748b'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:07.978171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-99ea1fcc-038c-4e36-ac26-e3bcb785748b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df7627d189c348a7aa383fca5a61e5d1', 'request_id': 'req-99ea1fcc-038c-4e36-ac26-e3bcb785748b'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:07.989151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-99ea1fcc-038c-4e36-ac26-e3bcb785748b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df7627d189c348a7aa383fca5a61e5d1', 'request_id': 'req-99ea1fcc-038c-4e36-ac26-e3bcb785748b'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:08.003937 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-99ea1fcc-038c-4e36-ac26-e3bcb785748b admin admin] The token cache is being invalidate because project df7627d189c348a7aa383fca5a61e5d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:28:08.004268 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-99ea1fcc-038c-4e36-ac26-e3bcb785748b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:08.004970 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-99ea1fcc-038c-4e36-ac26-e3bcb785748b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:08.006506 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 954/1913] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:28:07 2022] DELETE /identity/v3/projects/df7627d189c348a7aa383fca5a61e5d1 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:08.010602 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24550071-4e49-4c4e-9580-5aaf607e5733 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:08.022640 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24550071-4e49-4c4e-9580-5aaf607e5733 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:08.048330 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:08.089424 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6cdea4-1f88-41ec-94c7-4bec5c7801a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:08.090072 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6cdea4-1f88-41ec-94c7-4bec5c7801a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:08.090353 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6cdea4-1f88-41ec-94c7-4bec5c7801a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:08.090623 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6cdea4-1f88-41ec-94c7-4bec5c7801a9 admin admin] PATH_INFO: `/v3/users/e9a77c678ee44d7f8cdce1226915682b` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:08.091942 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6cdea4-1f88-41ec-94c7-4bec5c7801a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9a77c678ee44d7f8cdce1226915682b)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:08.094067 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6cdea4-1f88-41ec-94c7-4bec5c7801a9 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:08.147547 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-de6cdea4-1f88-41ec-94c7-4bec5c7801a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9a77c678ee44d7f8cdce1226915682b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:08.148406 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-de6cdea4-1f88-41ec-94c7-4bec5c7801a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9a77c678ee44d7f8cdce1226915682b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:08.158035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-de6cdea4-1f88-41ec-94c7-4bec5c7801a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9a77c678ee44d7f8cdce1226915682b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:08.173012 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-de6cdea4-1f88-41ec-94c7-4bec5c7801a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9a77c678ee44d7f8cdce1226915682b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:08.180476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-de6cdea4-1f88-41ec-94c7-4bec5c7801a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9a77c678ee44d7f8cdce1226915682b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:08.186543 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 960/1914] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:28:08 2022] DELETE /identity/v3/users/e9a77c678ee44d7f8cdce1226915682b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:08.483222 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea1fcc-038c-4e36-ac26-e3bcb785748b admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:08.485246 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-99ea1fcc-038c-4e36-ac26-e3bcb785748b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:08.501480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea1fcc-038c-4e36-ac26-e3bcb785748b admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:08.579343 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434fe16-4892-4ef2-9fc4-55513844c807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:08.580003 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434fe16-4892-4ef2-9fc4-55513844c807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:08.580287 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434fe16-4892-4ef2-9fc4-55513844c807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:08.580561 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434fe16-4892-4ef2-9fc4-55513844c807 admin admin] PATH_INFO: `/v3/projects/cef43e27688e44d1bcc6967cf7d33321` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:08.581811 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7434fe16-4892-4ef2-9fc4-55513844c807 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cef43e27688e44d1bcc6967cf7d33321)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:08.584064 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7434fe16-4892-4ef2-9fc4-55513844c807 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:08.642644 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-7434fe16-4892-4ef2-9fc4-55513844c807 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cef43e27688e44d1bcc6967cf7d33321', 'request_id': 'req-7434fe16-4892-4ef2-9fc4-55513844c807'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:08.649054 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-7434fe16-4892-4ef2-9fc4-55513844c807 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cef43e27688e44d1bcc6967cf7d33321', 'request_id': 'req-7434fe16-4892-4ef2-9fc4-55513844c807'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:08.657810 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-7434fe16-4892-4ef2-9fc4-55513844c807 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cef43e27688e44d1bcc6967cf7d33321', 'request_id': 'req-7434fe16-4892-4ef2-9fc4-55513844c807'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:08.670837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-7434fe16-4892-4ef2-9fc4-55513844c807 admin admin] The token cache is being invalidate because project cef43e27688e44d1bcc6967cf7d33321 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:28:08.671058 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-7434fe16-4892-4ef2-9fc4-55513844c807 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:08.671723 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-7434fe16-4892-4ef2-9fc4-55513844c807 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:08.673273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 955/1915] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:28:08 2022] DELETE /identity/v3/projects/cef43e27688e44d1bcc6967cf7d33321 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:28:08.678341 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040682ce-0027-457f-83f4-bce63d4ff0c5 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:08.678907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040682ce-0027-457f-83f4-bce63d4ff0c5 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:08.679012 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040682ce-0027-457f-83f4-bce63d4ff0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:08.700692 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-040682ce-0027-457f-83f4-bce63d4ff0c5 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:08.701934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-040682ce-0027-457f-83f4-bce63d4ff0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:08.792073 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 961/1916] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:08.800967 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a64d2-3e3f-4f2c-a8f2-3851499c874a None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:08.801366 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a64d2-3e3f-4f2c-a8f2-3851499c874a None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:08.801576 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a64d2-3e3f-4f2c-a8f2-3851499c874a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:08.829750 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-f96a64d2-3e3f-4f2c-a8f2-3851499c874a None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:08.831000 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-f96a64d2-3e3f-4f2c-a8f2-3851499c874a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:08.839526 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 956/1917] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:28:08.844334 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040682ce-0027-457f-83f4-bce63d4ff0c5 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:08.857355 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040682ce-0027-457f-83f4-bce63d4ff0c5 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:08.862683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6211fe-a462-4474-8e0a-f815ac3d811e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:08.863309 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6211fe-a462-4474-8e0a-f815ac3d811e admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:08.863590 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6211fe-a462-4474-8e0a-f815ac3d811e admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:08.863860 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6211fe-a462-4474-8e0a-f815ac3d811e admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:08.865044 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6211fe-a462-4474-8e0a-f815ac3d811e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:28:08.865355 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6211fe-a462-4474-8e0a-f815ac3d811e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:08.866969 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6211fe-a462-4474-8e0a-f815ac3d811e admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:08.888491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 962/1918] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:28:08.889465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a64d2-3e3f-4f2c-a8f2-3851499c874a None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:08.906183 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a64d2-3e3f-4f2c-a8f2-3851499c874a None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:08.911821 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4659e-7b03-4f24-9cfb-f3102b312de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:08.912529 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4659e-7b03-4f24-9cfb-f3102b312de3 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:08.912936 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4659e-7b03-4f24-9cfb-f3102b312de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:08.913410 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4659e-7b03-4f24-9cfb-f3102b312de3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:08.914361 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f4659e-7b03-4f24-9cfb-f3102b312de3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:08.916838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f4659e-7b03-4f24-9cfb-f3102b312de3 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:08.922434 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:08.951960 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 957/1919] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:28:08 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:28:08.954375 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6211fe-a462-4474-8e0a-f815ac3d811e admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:08.966021 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6211fe-a462-4474-8e0a-f815ac3d811e admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:08.994181 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:09.028952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaece127-aba7-46c6-896c-283f82767a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:09.029614 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaece127-aba7-46c6-896c-283f82767a6e admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:09.029823 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaece127-aba7-46c6-896c-283f82767a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:09.030091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaece127-aba7-46c6-896c-283f82767a6e admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:09.030711 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaece127-aba7-46c6-896c-283f82767a6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:09.032690 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaece127-aba7-46c6-896c-283f82767a6e admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:09.056779 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 963/1920] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:28:08 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:28:09.067938 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4659e-7b03-4f24-9cfb-f3102b312de3 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:09.079027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4659e-7b03-4f24-9cfb-f3102b312de3 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:09.083801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb35404-4664-4844-9993-248e10e958f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:09.084516 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb35404-4664-4844-9993-248e10e958f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:09.084858 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb35404-4664-4844-9993-248e10e958f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:09.085212 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb35404-4664-4844-9993-248e10e958f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:09.085932 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb35404-4664-4844-9993-248e10e958f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:09.087625 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb35404-4664-4844-9993-248e10e958f0 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:09.098787 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 958/1921] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:09.102847 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaece127-aba7-46c6-896c-283f82767a6e admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:09.113418 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaece127-aba7-46c6-896c-283f82767a6e admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:09.118254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d68027c-ec01-4863-bc2f-14a5653a6b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:09.118729 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d68027c-ec01-4863-bc2f-14a5653a6b37 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:09.119002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d68027c-ec01-4863-bc2f-14a5653a6b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:09.119269 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d68027c-ec01-4863-bc2f-14a5653a6b37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:09.119925 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d68027c-ec01-4863-bc2f-14a5653a6b37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:09.121544 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d68027c-ec01-4863-bc2f-14a5653a6b37 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:09.132729 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 964/1922] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:09.136912 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb35404-4664-4844-9993-248e10e958f0 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:09.147125 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb35404-4664-4844-9993-248e10e958f0 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:09.151635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4168922-ff4b-411b-bd4e-db4136978354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:09.152318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4168922-ff4b-411b-bd4e-db4136978354 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:09.152594 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4168922-ff4b-411b-bd4e-db4136978354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:09.152863 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4168922-ff4b-411b-bd4e-db4136978354 admin admin] PATH_INFO: `/v3/projects/22ebc63c0db84f908cfcded78a3c1c69/users/98dd70d4732c4bb9a86f2307fb67af2c/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:09.169152 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4168922-ff4b-411b-bd4e-db4136978354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22ebc63c0db84f908cfcded78a3c1c69, user_id=98dd70d4732c4bb9a86f2307fb67af2c, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:09.171892 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4168922-ff4b-411b-bd4e-db4136978354 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:09.186831 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 959/1923] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:28:09 2022] PUT /identity/v3/projects/22ebc63c0db84f908cfcded78a3c1c69/users/98dd70d4732c4bb9a86f2307fb67af2c/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:09.197241 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77babc1f-1152-4d88-9dae-479acc4e0fd7 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:09.197530 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77babc1f-1152-4d88-9dae-479acc4e0fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:09.197805 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77babc1f-1152-4d88-9dae-479acc4e0fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:09.232961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-77babc1f-1152-4d88-9dae-479acc4e0fd7 None None] MFA Rules not processed for user `98dd70d4732c4bb9a86f2307fb67af2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:09.234300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-77babc1f-1152-4d88-9dae-479acc4e0fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:09.322919 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 965/1924] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:09.327690 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4168922-ff4b-411b-bd4e-db4136978354 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:09.391987 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4168922-ff4b-411b-bd4e-db4136978354 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:09.396514 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a33a648-6045-4951-82ca-f8b3714a0d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:09.397120 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33a648-6045-4951-82ca-f8b3714a0d50 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:09.397396 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33a648-6045-4951-82ca-f8b3714a0d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:09.397666 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33a648-6045-4951-82ca-f8b3714a0d50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:09.398278 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a33a648-6045-4951-82ca-f8b3714a0d50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:09.400196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a33a648-6045-4951-82ca-f8b3714a0d50 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:09.434756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 960/1925] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:28:09 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:28:09.439221 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77babc1f-1152-4d88-9dae-479acc4e0fd7 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:09.450554 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77babc1f-1152-4d88-9dae-479acc4e0fd7 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:09.521084 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9b0f9-9646-42fd-abae-2d1f98557fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:09.522093 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9b0f9-9646-42fd-abae-2d1f98557fbb admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:09.522542 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9b0f9-9646-42fd-abae-2d1f98557fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:09.522962 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9b0f9-9646-42fd-abae-2d1f98557fbb admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:09.524070 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9b0f9-9646-42fd-abae-2d1f98557fbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:09.527178 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9b0f9-9646-42fd-abae-2d1f98557fbb admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:09.552649 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 966/1926] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:28:09 2022] POST /identity/v3/users => generated 298 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:28:09.556882 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a33a648-6045-4951-82ca-f8b3714a0d50 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:09.568184 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a33a648-6045-4951-82ca-f8b3714a0d50 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:09.573446 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30fe3a-5651-41df-b770-68dcea26ea16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:09.574053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30fe3a-5651-41df-b770-68dcea26ea16 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:09.574331 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30fe3a-5651-41df-b770-68dcea26ea16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:09.574605 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30fe3a-5651-41df-b770-68dcea26ea16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:09.575250 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30fe3a-5651-41df-b770-68dcea26ea16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:09.576896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30fe3a-5651-41df-b770-68dcea26ea16 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:09.588579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 961/1927] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:09.592946 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9b0f9-9646-42fd-abae-2d1f98557fbb admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:09.604263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9b0f9-9646-42fd-abae-2d1f98557fbb admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:09.610218 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c987f67-3367-4fab-a869-9c3a7be7820b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:09.610868 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c987f67-3367-4fab-a869-9c3a7be7820b admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:09.611144 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c987f67-3367-4fab-a869-9c3a7be7820b admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:09.611412 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c987f67-3367-4fab-a869-9c3a7be7820b admin admin] PATH_INFO: `/v3/projects/638326cec7ea477abc8895ff30566780/users/4a933f4c3b84496abb7a174df0c81d90/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:09.628721 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c987f67-3367-4fab-a869-9c3a7be7820b admin admin] RBAC: Authorizing `identity:create_grant(project_id=638326cec7ea477abc8895ff30566780, user_id=4a933f4c3b84496abb7a174df0c81d90, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:09.631576 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c987f67-3367-4fab-a869-9c3a7be7820b admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:09.647385 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 967/1928] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:28:09 2022] PUT /identity/v3/projects/638326cec7ea477abc8895ff30566780/users/4a933f4c3b84496abb7a174df0c81d90/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:09.653665 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30fe3a-5651-41df-b770-68dcea26ea16 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:09.664150 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30fe3a-5651-41df-b770-68dcea26ea16 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:09.725054 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d984d1-fcfd-4171-93d9-2acfd486ea71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:09.725684 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d984d1-fcfd-4171-93d9-2acfd486ea71 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:09.725965 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d984d1-fcfd-4171-93d9-2acfd486ea71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:09.726237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d984d1-fcfd-4171-93d9-2acfd486ea71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:09.726884 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d984d1-fcfd-4171-93d9-2acfd486ea71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:09.728525 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d984d1-fcfd-4171-93d9-2acfd486ea71 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:09.739418 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 962/1929] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:09.743950 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c987f67-3367-4fab-a869-9c3a7be7820b admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:09.749631 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:09.756352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c987f67-3367-4fab-a869-9c3a7be7820b admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:09.761728 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd598be-1146-427f-99d0-3095b2445b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:09.762488 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd598be-1146-427f-99d0-3095b2445b13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:09.762772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd598be-1146-427f-99d0-3095b2445b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:09.763044 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd598be-1146-427f-99d0-3095b2445b13 admin admin] PATH_INFO: `/v3/projects/638326cec7ea477abc8895ff30566780/users/4a933f4c3b84496abb7a174df0c81d90/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:09.764968 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd598be-1146-427f-99d0-3095b2445b13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=638326cec7ea477abc8895ff30566780, user_id=4a933f4c3b84496abb7a174df0c81d90, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:09.768396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd598be-1146-427f-99d0-3095b2445b13 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:09.779243 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 968/1930] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:28:09 2022] PUT /identity/v3/projects/638326cec7ea477abc8895ff30566780/users/4a933f4c3b84496abb7a174df0c81d90/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:09.789916 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ed2a2-179c-42ef-9664-2c67cd0fe1f2 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:09.790215 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ed2a2-179c-42ef-9664-2c67cd0fe1f2 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:09.790523 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ed2a2-179c-42ef-9664-2c67cd0fe1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:09.823466 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-649ed2a2-179c-42ef-9664-2c67cd0fe1f2 None None] MFA Rules not processed for user `4a933f4c3b84496abb7a174df0c81d90`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:09.824802 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-649ed2a2-179c-42ef-9664-2c67cd0fe1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:09.833943 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:09.895875 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 963/1931] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:09 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:09.908389 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd598be-1146-427f-99d0-3095b2445b13 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:09.911417 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-8dd598be-1146-427f-99d0-3095b2445b13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:09.982741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd598be-1146-427f-99d0-3095b2445b13 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:09.987054 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30230a0-b6b9-4a29-8e24-37b3bbe87ea0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:09.988117 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30230a0-b6b9-4a29-8e24-37b3bbe87ea0 service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:09.988405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30230a0-b6b9-4a29-8e24-37b3bbe87ea0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:09.988671 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30230a0-b6b9-4a29-8e24-37b3bbe87ea0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:10.001369 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30230a0-b6b9-4a29-8e24-37b3bbe87ea0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:10.003320 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30230a0-b6b9-4a29-8e24-37b3bbe87ea0 service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:10.016211 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 969/1932] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:28:09 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:28:10.115802 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ed2a2-179c-42ef-9664-2c67cd0fe1f2 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:10.128863 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ed2a2-179c-42ef-9664-2c67cd0fe1f2 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:10.133077 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6c2a2-7181-48db-bf33-e472e7501569 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:10.133718 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6c2a2-7181-48db-bf33-e472e7501569 service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:10.134021 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6c2a2-7181-48db-bf33-e472e7501569 service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:10.134325 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6c2a2-7181-48db-bf33-e472e7501569 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:10.199046 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef6c2a2-7181-48db-bf33-e472e7501569 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:10.200696 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef6c2a2-7181-48db-bf33-e472e7501569 service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:10.212536 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 964/1933] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:28:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:28:10.636086 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30230a0-b6b9-4a29-8e24-37b3bbe87ea0 service cinder] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:10.709249 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30230a0-b6b9-4a29-8e24-37b3bbe87ea0 service cinder] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:10.714274 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d6e88a-f391-4083-bfa5-b5c389b0656a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:10.714994 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6e88a-f391-4083-bfa5-b5c389b0656a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:10.715382 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6e88a-f391-4083-bfa5-b5c389b0656a admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:10.715698 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6e88a-f391-4083-bfa5-b5c389b0656a admin admin] PATH_INFO: `/v3/users/98dd70d4732c4bb9a86f2307fb67af2c` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:10.716910 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d6e88a-f391-4083-bfa5-b5c389b0656a admin admin] RBAC: Authorizing `identity:delete_user(user_id=98dd70d4732c4bb9a86f2307fb67af2c)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:10.718876 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d6e88a-f391-4083-bfa5-b5c389b0656a admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:10.778023 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a8d6e88a-f391-4083-bfa5-b5c389b0656a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98dd70d4732c4bb9a86f2307fb67af2c'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:10.779086 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a8d6e88a-f391-4083-bfa5-b5c389b0656a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98dd70d4732c4bb9a86f2307fb67af2c'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:10.788510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a8d6e88a-f391-4083-bfa5-b5c389b0656a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98dd70d4732c4bb9a86f2307fb67af2c'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:10.803154 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a8d6e88a-f391-4083-bfa5-b5c389b0656a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98dd70d4732c4bb9a86f2307fb67af2c'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:10.810840 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a8d6e88a-f391-4083-bfa5-b5c389b0656a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98dd70d4732c4bb9a86f2307fb67af2c'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:10.818067 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 970/1934] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:28:10 2022] DELETE /identity/v3/users/98dd70d4732c4bb9a86f2307fb67af2c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:10.822487 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbe122-86fd-402c-9579-9f8f31eb76fb None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:10.822799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbe122-86fd-402c-9579-9f8f31eb76fb None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:10.823074 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbe122-86fd-402c-9579-9f8f31eb76fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:10.845767 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-85dbe122-86fd-402c-9579-9f8f31eb76fb None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:10.847282 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-85dbe122-86fd-402c-9579-9f8f31eb76fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:10.935865 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 965/1935] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:10.943053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d6e88a-f391-4083-bfa5-b5c389b0656a admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:10.945498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a8d6e88a-f391-4083-bfa5-b5c389b0656a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:10.961097 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:11.009780 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d6e88a-f391-4083-bfa5-b5c389b0656a admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:11.013291 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535debf-43d0-41fe-887c-8d2fadde496f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:11.013901 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535debf-43d0-41fe-887c-8d2fadde496f service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:11.014180 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535debf-43d0-41fe-887c-8d2fadde496f service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:11.014451 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535debf-43d0-41fe-887c-8d2fadde496f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:11.023464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535debf-43d0-41fe-887c-8d2fadde496f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:11.024990 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535debf-43d0-41fe-887c-8d2fadde496f service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:11.034431 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 971/1936] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:28:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:28:11.388111 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbe122-86fd-402c-9579-9f8f31eb76fb None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:11.389773 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-85dbe122-86fd-402c-9579-9f8f31eb76fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:11.403703 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbe122-86fd-402c-9579-9f8f31eb76fb None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:11.407761 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1c613-c773-4b46-9b0d-09405e890469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:11.408772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1c613-c773-4b46-9b0d-09405e890469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:11.408941 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1c613-c773-4b46-9b0d-09405e890469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:11.409171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1c613-c773-4b46-9b0d-09405e890469 admin admin] PATH_INFO: `/v3/projects/22ebc63c0db84f908cfcded78a3c1c69` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:11.410357 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1c613-c773-4b46-9b0d-09405e890469 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22ebc63c0db84f908cfcded78a3c1c69)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:11.412584 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1c613-c773-4b46-9b0d-09405e890469 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:11.461281 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:11.476692 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-28e1c613-c773-4b46-9b0d-09405e890469 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22ebc63c0db84f908cfcded78a3c1c69', 'request_id': 'req-28e1c613-c773-4b46-9b0d-09405e890469'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:11.484416 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-28e1c613-c773-4b46-9b0d-09405e890469 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22ebc63c0db84f908cfcded78a3c1c69', 'request_id': 'req-28e1c613-c773-4b46-9b0d-09405e890469'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:11.494183 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-28e1c613-c773-4b46-9b0d-09405e890469 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22ebc63c0db84f908cfcded78a3c1c69', 'request_id': 'req-28e1c613-c773-4b46-9b0d-09405e890469'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:11.508626 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-28e1c613-c773-4b46-9b0d-09405e890469 admin admin] The token cache is being invalidate because project 22ebc63c0db84f908cfcded78a3c1c69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:28:11.508931 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-28e1c613-c773-4b46-9b0d-09405e890469 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:11.509677 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-28e1c613-c773-4b46-9b0d-09405e890469 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:11.512074 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 966/1937] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:28:11 2022] DELETE /identity/v3/projects/22ebc63c0db84f908cfcded78a3c1c69 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:11.516820 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535debf-43d0-41fe-887c-8d2fadde496f service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:11.530042 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535debf-43d0-41fe-887c-8d2fadde496f service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:11.598571 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c5354-6f70-4dc0-87de-b1cbaefcbc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:11.599341 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c5354-6f70-4dc0-87de-b1cbaefcbc73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:11.599741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c5354-6f70-4dc0-87de-b1cbaefcbc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:11.600048 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c5354-6f70-4dc0-87de-b1cbaefcbc73 admin admin] PATH_INFO: `/v3/users/4a933f4c3b84496abb7a174df0c81d90` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:11.601248 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7c5354-6f70-4dc0-87de-b1cbaefcbc73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a933f4c3b84496abb7a174df0c81d90)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:11.603094 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7c5354-6f70-4dc0-87de-b1cbaefcbc73 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:11.652305 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8c7c5354-6f70-4dc0-87de-b1cbaefcbc73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a933f4c3b84496abb7a174df0c81d90'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:11.652963 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8c7c5354-6f70-4dc0-87de-b1cbaefcbc73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a933f4c3b84496abb7a174df0c81d90'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:11.660833 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8c7c5354-6f70-4dc0-87de-b1cbaefcbc73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a933f4c3b84496abb7a174df0c81d90'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:11.675043 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8c7c5354-6f70-4dc0-87de-b1cbaefcbc73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a933f4c3b84496abb7a174df0c81d90'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:11.683472 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8c7c5354-6f70-4dc0-87de-b1cbaefcbc73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a933f4c3b84496abb7a174df0c81d90'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:11.690452 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 972/1938] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:28:11 2022] DELETE /identity/v3/users/4a933f4c3b84496abb7a174df0c81d90 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:12.271692 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1c613-c773-4b46-9b0d-09405e890469 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:12.273957 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-28e1c613-c773-4b46-9b0d-09405e890469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:12.301765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1c613-c773-4b46-9b0d-09405e890469 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:12.379377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279cb75b-242e-45c3-8f60-7d3cfcfc1088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:12.381433 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cb75b-242e-45c3-8f60-7d3cfcfc1088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:12.382028 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cb75b-242e-45c3-8f60-7d3cfcfc1088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:12.382721 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cb75b-242e-45c3-8f60-7d3cfcfc1088 admin admin] PATH_INFO: `/v3/projects/638326cec7ea477abc8895ff30566780` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:12.386434 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279cb75b-242e-45c3-8f60-7d3cfcfc1088 admin admin] RBAC: Authorizing `identity:delete_project(project_id=638326cec7ea477abc8895ff30566780)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:12.389441 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279cb75b-242e-45c3-8f60-7d3cfcfc1088 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:12.442583 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-279cb75b-242e-45c3-8f60-7d3cfcfc1088 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '638326cec7ea477abc8895ff30566780', 'request_id': 'req-279cb75b-242e-45c3-8f60-7d3cfcfc1088'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:12.450231 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-279cb75b-242e-45c3-8f60-7d3cfcfc1088 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '638326cec7ea477abc8895ff30566780', 'request_id': 'req-279cb75b-242e-45c3-8f60-7d3cfcfc1088'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:12.465116 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-279cb75b-242e-45c3-8f60-7d3cfcfc1088 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '638326cec7ea477abc8895ff30566780', 'request_id': 'req-279cb75b-242e-45c3-8f60-7d3cfcfc1088'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:12.477774 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-279cb75b-242e-45c3-8f60-7d3cfcfc1088 admin admin] The token cache is being invalidate because project 638326cec7ea477abc8895ff30566780 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:28:12.478090 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-279cb75b-242e-45c3-8f60-7d3cfcfc1088 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:12.478780 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-279cb75b-242e-45c3-8f60-7d3cfcfc1088 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:12.480436 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 967/1939] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:28:12 2022] DELETE /identity/v3/projects/638326cec7ea477abc8895ff30566780 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:12.486178 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58140c0c-50d9-4c7d-bd4f-542b206ad638 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:12.486564 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58140c0c-50d9-4c7d-bd4f-542b206ad638 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:12.486969 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58140c0c-50d9-4c7d-bd4f-542b206ad638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:12.507063 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-58140c0c-50d9-4c7d-bd4f-542b206ad638 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:12.508301 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-58140c0c-50d9-4c7d-bd4f-542b206ad638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:12.539237 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:12.581590 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 973/1940] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:12.589596 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c27866-8d2a-474d-b762-c848533bb254 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:12.589890 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c27866-8d2a-474d-b762-c848533bb254 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:12.590195 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c27866-8d2a-474d-b762-c848533bb254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:12.610878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-22c27866-8d2a-474d-b762-c848533bb254 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:12.612136 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-22c27866-8d2a-474d-b762-c848533bb254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:12.621683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 968/1941] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:28:12.625497 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58140c0c-50d9-4c7d-bd4f-542b206ad638 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:12.635755 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58140c0c-50d9-4c7d-bd4f-542b206ad638 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:12.640642 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33b555-2624-471a-848a-c5d78747e209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:12.641259 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33b555-2624-471a-848a-c5d78747e209 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:12.641544 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33b555-2624-471a-848a-c5d78747e209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:12.641813 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33b555-2624-471a-848a-c5d78747e209 admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:12.642468 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c33b555-2624-471a-848a-c5d78747e209 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:28:12.642777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c33b555-2624-471a-848a-c5d78747e209 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:12.644421 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c33b555-2624-471a-848a-c5d78747e209 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:12.657575 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 974/1942] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:28:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:28:12.661901 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c27866-8d2a-474d-b762-c848533bb254 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:12.671201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c27866-8d2a-474d-b762-c848533bb254 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:12.676327 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d134eb-e64b-4f69-a8cb-895c848a3edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:12.676964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d134eb-e64b-4f69-a8cb-895c848a3edc admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:12.677231 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d134eb-e64b-4f69-a8cb-895c848a3edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:12.677505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d134eb-e64b-4f69-a8cb-895c848a3edc admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:12.678165 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d134eb-e64b-4f69-a8cb-895c848a3edc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:12.680126 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d134eb-e64b-4f69-a8cb-895c848a3edc admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:12.708725 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 969/1943] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:28:12 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:28:12.712973 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33b555-2624-471a-848a-c5d78747e209 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:12.723571 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33b555-2624-471a-848a-c5d78747e209 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:12.791723 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e474442-b188-4169-9a65-e0965c5a3567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:12.792364 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e474442-b188-4169-9a65-e0965c5a3567 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:12.792642 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e474442-b188-4169-9a65-e0965c5a3567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:12.792911 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e474442-b188-4169-9a65-e0965c5a3567 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:12.793581 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e474442-b188-4169-9a65-e0965c5a3567 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:12.795715 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e474442-b188-4169-9a65-e0965c5a3567 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:12.818367 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 975/1944] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:28:12 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:28:12.822494 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d134eb-e64b-4f69-a8cb-895c848a3edc admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:12.832956 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d134eb-e64b-4f69-a8cb-895c848a3edc admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:12.837865 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa18b9-11a3-40d1-bd14-cf29306928f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:12.838589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa18b9-11a3-40d1-bd14-cf29306928f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:12.838866 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa18b9-11a3-40d1-bd14-cf29306928f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:12.839153 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa18b9-11a3-40d1-bd14-cf29306928f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:12.839825 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa18b9-11a3-40d1-bd14-cf29306928f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:12.841457 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa18b9-11a3-40d1-bd14-cf29306928f2 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:12.852871 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 970/1945] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:12.856708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e474442-b188-4169-9a65-e0965c5a3567 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:12.867132 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e474442-b188-4169-9a65-e0965c5a3567 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:12.871865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf87ea7-0ab2-45c2-8c92-58eacb7b0f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:12.872532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf87ea7-0ab2-45c2-8c92-58eacb7b0f77 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:12.872811 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf87ea7-0ab2-45c2-8c92-58eacb7b0f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:12.873082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf87ea7-0ab2-45c2-8c92-58eacb7b0f77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:12.873733 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf87ea7-0ab2-45c2-8c92-58eacb7b0f77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:12.875383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf87ea7-0ab2-45c2-8c92-58eacb7b0f77 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:12.886679 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 976/1946] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:12.891052 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa18b9-11a3-40d1-bd14-cf29306928f2 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:12.901046 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa18b9-11a3-40d1-bd14-cf29306928f2 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:12.906072 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f92a8-bfe3-4a9a-949d-301eba9d56e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:12.906729 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f92a8-bfe3-4a9a-949d-301eba9d56e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:12.907008 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f92a8-bfe3-4a9a-949d-301eba9d56e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:12.907274 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f92a8-bfe3-4a9a-949d-301eba9d56e9 admin admin] PATH_INFO: `/v3/projects/3313c17c296b4abeb801affc678455fc/users/07159bf81384428386943b3d0af7e5b8/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:12.921805 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f92a8-bfe3-4a9a-949d-301eba9d56e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3313c17c296b4abeb801affc678455fc, user_id=07159bf81384428386943b3d0af7e5b8, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:12.924532 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f92a8-bfe3-4a9a-949d-301eba9d56e9 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:12.926782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:12.940490 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 971/1947] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:28:12 2022] PUT /identity/v3/projects/3313c17c296b4abeb801affc678455fc/users/07159bf81384428386943b3d0af7e5b8/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:12.953608 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8fc9d9-41d2-40b4-9ca9-7492f582616c None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:12.953922 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8fc9d9-41d2-40b4-9ca9-7492f582616c None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:12.954200 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8fc9d9-41d2-40b4-9ca9-7492f582616c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:12.985749 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-3d8fc9d9-41d2-40b4-9ca9-7492f582616c None None] MFA Rules not processed for user `07159bf81384428386943b3d0af7e5b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:12.987079 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-3d8fc9d9-41d2-40b4-9ca9-7492f582616c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:13.064514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 977/1948] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:13.075148 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f92a8-bfe3-4a9a-949d-301eba9d56e9 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:13.076897 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-564f92a8-bfe3-4a9a-949d-301eba9d56e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:13.138975 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f92a8-bfe3-4a9a-949d-301eba9d56e9 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:13.142760 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ea583-00b4-4a7d-86be-f87aee029d50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:13.143455 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ea583-00b4-4a7d-86be-f87aee029d50 service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:13.143791 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ea583-00b4-4a7d-86be-f87aee029d50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:13.144149 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ea583-00b4-4a7d-86be-f87aee029d50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:13.152683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ea583-00b4-4a7d-86be-f87aee029d50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:13.154147 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ea583-00b4-4a7d-86be-f87aee029d50 service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:13.163744 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 972/1949] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:28:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:28:23.060949 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:23.159845 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:24.923861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8fc9d9-41d2-40b4-9ca9-7492f582616c None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:25.019263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8fc9d9-41d2-40b4-9ca9-7492f582616c None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:25.024779 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6af83e-83be-472a-9ebc-548938a9eec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:25.025384 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6af83e-83be-472a-9ebc-548938a9eec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:25.025699 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6af83e-83be-472a-9ebc-548938a9eec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:25.025977 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6af83e-83be-472a-9ebc-548938a9eec0 admin admin] PATH_INFO: `/v3/users/07159bf81384428386943b3d0af7e5b8` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:25.027358 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6af83e-83be-472a-9ebc-548938a9eec0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07159bf81384428386943b3d0af7e5b8)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:25.029407 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6af83e-83be-472a-9ebc-548938a9eec0 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:25.129486 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-0b6af83e-83be-472a-9ebc-548938a9eec0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07159bf81384428386943b3d0af7e5b8'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:25.130710 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-0b6af83e-83be-472a-9ebc-548938a9eec0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07159bf81384428386943b3d0af7e5b8'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:25.142711 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-0b6af83e-83be-472a-9ebc-548938a9eec0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07159bf81384428386943b3d0af7e5b8'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:25.179098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-0b6af83e-83be-472a-9ebc-548938a9eec0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07159bf81384428386943b3d0af7e5b8'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:25.228908 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-0b6af83e-83be-472a-9ebc-548938a9eec0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07159bf81384428386943b3d0af7e5b8'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:25.239709 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 978/1950] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:28:24 2022] DELETE /identity/v3/users/07159bf81384428386943b3d0af7e5b8 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:25.244139 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a2da5-a6d3-4711-ab49-e9c0ece633d7 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:25.244691 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a2da5-a6d3-4711-ab49-e9c0ece633d7 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:25.244930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a2da5-a6d3-4711-ab49-e9c0ece633d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:25.270171 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-eb4a2da5-a6d3-4711-ab49-e9c0ece633d7 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:25.271646 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-eb4a2da5-a6d3-4711-ab49-e9c0ece633d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:25.352371 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 973/1951] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:25.362668 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6af83e-83be-472a-9ebc-548938a9eec0 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:25.364227 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-0b6af83e-83be-472a-9ebc-548938a9eec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:25.455285 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6af83e-83be-472a-9ebc-548938a9eec0 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:25.458101 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27efab0-3e49-46a3-b2b8-4450236a7d1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:25.458735 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27efab0-3e49-46a3-b2b8-4450236a7d1e service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:25.459022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27efab0-3e49-46a3-b2b8-4450236a7d1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:25.459295 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27efab0-3e49-46a3-b2b8-4450236a7d1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:25.471042 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27efab0-3e49-46a3-b2b8-4450236a7d1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:25.472864 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27efab0-3e49-46a3-b2b8-4450236a7d1e service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:25.489914 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 979/1952] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:28:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:28:25.869425 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a2da5-a6d3-4711-ab49-e9c0ece633d7 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:25.869831 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-eb4a2da5-a6d3-4711-ab49-e9c0ece633d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:25.914798 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a2da5-a6d3-4711-ab49-e9c0ece633d7 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:25.933292 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b781465-ad8c-4edb-b1a0-62f41f4a0212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:25.933668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b781465-ad8c-4edb-b1a0-62f41f4a0212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:25.934353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b781465-ad8c-4edb-b1a0-62f41f4a0212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:25.934778 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b781465-ad8c-4edb-b1a0-62f41f4a0212 admin admin] PATH_INFO: `/v3/projects/3313c17c296b4abeb801affc678455fc` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:25.936463 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b781465-ad8c-4edb-b1a0-62f41f4a0212 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3313c17c296b4abeb801affc678455fc)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:25.938787 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b781465-ad8c-4edb-b1a0-62f41f4a0212 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:26.008068 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-6b781465-ad8c-4edb-b1a0-62f41f4a0212 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3313c17c296b4abeb801affc678455fc', 'request_id': 'req-6b781465-ad8c-4edb-b1a0-62f41f4a0212'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:26.016881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-6b781465-ad8c-4edb-b1a0-62f41f4a0212 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3313c17c296b4abeb801affc678455fc', 'request_id': 'req-6b781465-ad8c-4edb-b1a0-62f41f4a0212'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:26.027396 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-6b781465-ad8c-4edb-b1a0-62f41f4a0212 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3313c17c296b4abeb801affc678455fc', 'request_id': 'req-6b781465-ad8c-4edb-b1a0-62f41f4a0212'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:26.044283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-6b781465-ad8c-4edb-b1a0-62f41f4a0212 admin admin] The token cache is being invalidate because project 3313c17c296b4abeb801affc678455fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:28:26.044708 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-6b781465-ad8c-4edb-b1a0-62f41f4a0212 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:26.045634 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-6b781465-ad8c-4edb-b1a0-62f41f4a0212 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:26.047628 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 974/1953] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:28:25 2022] DELETE /identity/v3/projects/3313c17c296b4abeb801affc678455fc => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:26.052893 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5bf07-3d36-4c11-a00c-0aa731f085f8 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:26.053406 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5bf07-3d36-4c11-a00c-0aa731f085f8 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:26.053729 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5bf07-3d36-4c11-a00c-0aa731f085f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:26.079851 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-77b5bf07-3d36-4c11-a00c-0aa731f085f8 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:26.081469 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-77b5bf07-3d36-4c11-a00c-0aa731f085f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:26.103908 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:26.161890 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 980/1954] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:26.171260 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8379cf-b17d-4157-a3b7-a13d485fa4a7 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:26.171856 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8379cf-b17d-4157-a3b7-a13d485fa4a7 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:26.172288 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8379cf-b17d-4157-a3b7-a13d485fa4a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:26.205662 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-1b8379cf-b17d-4157-a3b7-a13d485fa4a7 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:26.206886 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-1b8379cf-b17d-4157-a3b7-a13d485fa4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:26.216547 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 975/1955] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:26.221156 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b5bf07-3d36-4c11-a00c-0aa731f085f8 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:26.239602 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b5bf07-3d36-4c11-a00c-0aa731f085f8 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:26.247437 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fb3d3-8c39-4522-b698-5584ac871393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:26.248111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fb3d3-8c39-4522-b698-5584ac871393 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:26.248555 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fb3d3-8c39-4522-b698-5584ac871393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:26.249238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fb3d3-8c39-4522-b698-5584ac871393 admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:26.250380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4fb3d3-8c39-4522-b698-5584ac871393 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:28:26.250691 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4fb3d3-8c39-4522-b698-5584ac871393 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:26.261274 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4fb3d3-8c39-4522-b698-5584ac871393 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:26.280126 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 981/1956] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:28:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:28:26.284382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8379cf-b17d-4157-a3b7-a13d485fa4a7 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:26.294006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8379cf-b17d-4157-a3b7-a13d485fa4a7 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:26.299268 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07aa450-77e6-46c8-b635-50fe61d26031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:26.299948 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07aa450-77e6-46c8-b635-50fe61d26031 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:26.300199 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07aa450-77e6-46c8-b635-50fe61d26031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:26.300614 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07aa450-77e6-46c8-b635-50fe61d26031 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:26.301284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07aa450-77e6-46c8-b635-50fe61d26031 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:26.303260 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07aa450-77e6-46c8-b635-50fe61d26031 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:26.338957 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 976/1957] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:28:26 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:28:26.340572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fb3d3-8c39-4522-b698-5584ac871393 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:26.351509 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fb3d3-8c39-4522-b698-5584ac871393 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:26.423836 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da3176-7eb3-46cc-8171-6ecce1004a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:26.424764 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da3176-7eb3-46cc-8171-6ecce1004a4b admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:26.425065 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da3176-7eb3-46cc-8171-6ecce1004a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:26.425354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da3176-7eb3-46cc-8171-6ecce1004a4b admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:26.427000 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da3176-7eb3-46cc-8171-6ecce1004a4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:26.429120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da3176-7eb3-46cc-8171-6ecce1004a4b admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:26.465815 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 982/1958] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:28:26 2022] POST /identity/v3/users => generated 287 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:28:26.467018 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07aa450-77e6-46c8-b635-50fe61d26031 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:26.488908 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07aa450-77e6-46c8-b635-50fe61d26031 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:26.499363 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191a137-d336-418a-bb69-48122ee6b465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:26.500032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191a137-d336-418a-bb69-48122ee6b465 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:26.500312 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191a137-d336-418a-bb69-48122ee6b465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:26.500633 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191a137-d336-418a-bb69-48122ee6b465 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:26.501349 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a191a137-d336-418a-bb69-48122ee6b465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:26.503093 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a191a137-d336-418a-bb69-48122ee6b465 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:26.517626 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 977/1959] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:26.521840 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da3176-7eb3-46cc-8171-6ecce1004a4b admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:26.533159 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da3176-7eb3-46cc-8171-6ecce1004a4b admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:26.538787 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057762c8-ed29-40d5-a815-dc6cd51a17c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:26.539441 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057762c8-ed29-40d5-a815-dc6cd51a17c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:26.539720 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057762c8-ed29-40d5-a815-dc6cd51a17c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:26.540028 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057762c8-ed29-40d5-a815-dc6cd51a17c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:26.540713 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057762c8-ed29-40d5-a815-dc6cd51a17c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:26.542448 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057762c8-ed29-40d5-a815-dc6cd51a17c9 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:26.554853 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 983/1960] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:26.559123 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191a137-d336-418a-bb69-48122ee6b465 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:26.571566 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191a137-d336-418a-bb69-48122ee6b465 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:26.576271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd03358-6316-46de-b0e2-31a74e106eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:26.576950 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd03358-6316-46de-b0e2-31a74e106eb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:26.577232 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd03358-6316-46de-b0e2-31a74e106eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:26.577508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd03358-6316-46de-b0e2-31a74e106eb3 admin admin] PATH_INFO: `/v3/projects/1e98477f945c412c878234bbedfe7d4a/users/ea953da5d5294ddfaef2d18bbf8b98db/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:26.593323 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd03358-6316-46de-b0e2-31a74e106eb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e98477f945c412c878234bbedfe7d4a, user_id=ea953da5d5294ddfaef2d18bbf8b98db, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:26.596416 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd03358-6316-46de-b0e2-31a74e106eb3 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:26.613004 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 978/1961] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:28:26 2022] PUT /identity/v3/projects/1e98477f945c412c878234bbedfe7d4a/users/ea953da5d5294ddfaef2d18bbf8b98db/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:26.623527 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174966c-df85-4199-b2c7-446fdbe53751 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:26.623810 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174966c-df85-4199-b2c7-446fdbe53751 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:26.624117 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174966c-df85-4199-b2c7-446fdbe53751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:26.654518 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-3174966c-df85-4199-b2c7-446fdbe53751 None None] MFA Rules not processed for user `ea953da5d5294ddfaef2d18bbf8b98db`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:26.655737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-3174966c-df85-4199-b2c7-446fdbe53751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:26.730569 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 984/1962] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:26.743878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd03358-6316-46de-b0e2-31a74e106eb3 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:26.745725 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-7fd03358-6316-46de-b0e2-31a74e106eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:26.770352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:26.831366 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd03358-6316-46de-b0e2-31a74e106eb3 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:26.835150 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922b3ce-af8f-4dc7-b355-278bfd66ad6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:26.835771 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922b3ce-af8f-4dc7-b355-278bfd66ad6b service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:26.836092 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922b3ce-af8f-4dc7-b355-278bfd66ad6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:26.836364 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922b3ce-af8f-4dc7-b355-278bfd66ad6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:26.845821 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c922b3ce-af8f-4dc7-b355-278bfd66ad6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:26.847743 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c922b3ce-af8f-4dc7-b355-278bfd66ad6b service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:26.859434 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 979/1963] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:28:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:28:26.882467 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174966c-df85-4199-b2c7-446fdbe53751 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:26.905423 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:26.971790 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174966c-df85-4199-b2c7-446fdbe53751 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:26.979132 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8bdf26-5047-40be-bbb3-0dadd5521e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:26.980092 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8bdf26-5047-40be-bbb3-0dadd5521e16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:26.980585 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8bdf26-5047-40be-bbb3-0dadd5521e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:26.981007 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8bdf26-5047-40be-bbb3-0dadd5521e16 admin admin] PATH_INFO: `/v3/users/ea953da5d5294ddfaef2d18bbf8b98db` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:26.982382 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8bdf26-5047-40be-bbb3-0dadd5521e16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea953da5d5294ddfaef2d18bbf8b98db)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:26.984727 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8bdf26-5047-40be-bbb3-0dadd5521e16 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:27.050636 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-9d8bdf26-5047-40be-bbb3-0dadd5521e16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea953da5d5294ddfaef2d18bbf8b98db'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:27.052648 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-9d8bdf26-5047-40be-bbb3-0dadd5521e16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea953da5d5294ddfaef2d18bbf8b98db'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:27.066404 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-9d8bdf26-5047-40be-bbb3-0dadd5521e16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea953da5d5294ddfaef2d18bbf8b98db'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:27.110316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-9d8bdf26-5047-40be-bbb3-0dadd5521e16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea953da5d5294ddfaef2d18bbf8b98db'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:27.118927 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-9d8bdf26-5047-40be-bbb3-0dadd5521e16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea953da5d5294ddfaef2d18bbf8b98db'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:27.125661 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 985/1964] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:28:26 2022] DELETE /identity/v3/users/ea953da5d5294ddfaef2d18bbf8b98db => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:27.130450 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ed840-7dd4-481b-8844-3c50fa45da09 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:27.130737 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ed840-7dd4-481b-8844-3c50fa45da09 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:27.131010 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ed840-7dd4-481b-8844-3c50fa45da09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:27.150677 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-659ed840-7dd4-481b-8844-3c50fa45da09 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:27.151887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-659ed840-7dd4-481b-8844-3c50fa45da09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:27.220699 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 980/1965] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:28:27.231477 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8bdf26-5047-40be-bbb3-0dadd5521e16 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:27.233571 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-9d8bdf26-5047-40be-bbb3-0dadd5521e16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:27.296442 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8bdf26-5047-40be-bbb3-0dadd5521e16 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:27.300725 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd842936-0e45-43ad-93cf-72ddb23c0e08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:27.301348 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd842936-0e45-43ad-93cf-72ddb23c0e08 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:27.301628 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd842936-0e45-43ad-93cf-72ddb23c0e08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:27.301911 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd842936-0e45-43ad-93cf-72ddb23c0e08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:27.311741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd842936-0e45-43ad-93cf-72ddb23c0e08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:27.313566 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd842936-0e45-43ad-93cf-72ddb23c0e08 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:27.325371 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 986/1966] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:28:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:28:27.889755 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659ed840-7dd4-481b-8844-3c50fa45da09 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:27.893361 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-659ed840-7dd4-481b-8844-3c50fa45da09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:27.909717 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659ed840-7dd4-481b-8844-3c50fa45da09 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:27.915903 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be478c4-20d0-44ca-aa18-081f67d7e479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:27.916974 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be478c4-20d0-44ca-aa18-081f67d7e479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:27.916974 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be478c4-20d0-44ca-aa18-081f67d7e479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:27.917172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be478c4-20d0-44ca-aa18-081f67d7e479 admin admin] PATH_INFO: `/v3/users/b8a5935105e0406bb1e690096b5979d2` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:27.918952 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be478c4-20d0-44ca-aa18-081f67d7e479 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8a5935105e0406bb1e690096b5979d2)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:27.921810 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be478c4-20d0-44ca-aa18-081f67d7e479 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:27.987486 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-4be478c4-20d0-44ca-aa18-081f67d7e479 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8a5935105e0406bb1e690096b5979d2'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:28.003413 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-4be478c4-20d0-44ca-aa18-081f67d7e479 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8a5935105e0406bb1e690096b5979d2'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:28.043629 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-4be478c4-20d0-44ca-aa18-081f67d7e479 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8a5935105e0406bb1e690096b5979d2'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:28.044452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-4be478c4-20d0-44ca-aa18-081f67d7e479 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8a5935105e0406bb1e690096b5979d2'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:28.066672 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd842936-0e45-43ad-93cf-72ddb23c0e08 service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:28.068154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-4be478c4-20d0-44ca-aa18-081f67d7e479 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8a5935105e0406bb1e690096b5979d2'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:28.068368 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-dd842936-0e45-43ad-93cf-72ddb23c0e08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:28.087542 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 981/1967] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:28:27 2022] DELETE /identity/v3/users/b8a5935105e0406bb1e690096b5979d2 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:28.095800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd842936-0e45-43ad-93cf-72ddb23c0e08 service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:28.102622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8caec8-fbe9-4ef3-a8bc-d8a64aaf3d60 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:28.102913 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8caec8-fbe9-4ef3-a8bc-d8a64aaf3d60 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:28.104040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8caec8-fbe9-4ef3-a8bc-d8a64aaf3d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:28.135789 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-ea8caec8-fbe9-4ef3-a8bc-d8a64aaf3d60 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:28.137458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-ea8caec8-fbe9-4ef3-a8bc-d8a64aaf3d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:28.161108 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:28.176078 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:28.197405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9af8a-845d-4a04-aa63-60b92a4c453c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:28.198249 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9af8a-845d-4a04-aa63-60b92a4c453c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:28.198532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9af8a-845d-4a04-aa63-60b92a4c453c admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:28.198805 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9af8a-845d-4a04-aa63-60b92a4c453c admin admin] PATH_INFO: `/v3/projects/1e98477f945c412c878234bbedfe7d4a` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:28.200007 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e9af8a-845d-4a04-aa63-60b92a4c453c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e98477f945c412c878234bbedfe7d4a)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:28.201972 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e9af8a-845d-4a04-aa63-60b92a4c453c admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:28.215365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 982/1968] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:28:28.227618 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8caec8-fbe9-4ef3-a8bc-d8a64aaf3d60 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:28.229319 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-ea8caec8-fbe9-4ef3-a8bc-d8a64aaf3d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:28.259996 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-78e9af8a-845d-4a04-aa63-60b92a4c453c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e98477f945c412c878234bbedfe7d4a', 'request_id': 'req-78e9af8a-845d-4a04-aa63-60b92a4c453c'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:28.262815 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-78e9af8a-845d-4a04-aa63-60b92a4c453c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e98477f945c412c878234bbedfe7d4a', 'request_id': 'req-78e9af8a-845d-4a04-aa63-60b92a4c453c'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:28.271861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-78e9af8a-845d-4a04-aa63-60b92a4c453c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e98477f945c412c878234bbedfe7d4a', 'request_id': 'req-78e9af8a-845d-4a04-aa63-60b92a4c453c'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:28.287426 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-78e9af8a-845d-4a04-aa63-60b92a4c453c admin admin] The token cache is being invalidate because project 1e98477f945c412c878234bbedfe7d4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:28:28.287801 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-78e9af8a-845d-4a04-aa63-60b92a4c453c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:28.288488 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-78e9af8a-845d-4a04-aa63-60b92a4c453c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:28.290084 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 987/1969] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:28:28 2022] DELETE /identity/v3/projects/1e98477f945c412c878234bbedfe7d4a => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:28:28.294776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7b468-327a-474d-ae1f-f8695cbdcaed None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:28.295145 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7b468-327a-474d-ae1f-f8695cbdcaed None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:28.296104 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7b468-327a-474d-ae1f-f8695cbdcaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:28.298244 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8caec8-fbe9-4ef3-a8bc-d8a64aaf3d60 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:28.320761 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-59a7b468-327a-474d-ae1f-f8695cbdcaed None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:28.321607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-59a7b468-327a-474d-ae1f-f8695cbdcaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:28.360052 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2f17e-a4db-4627-9705-cabdb0937174 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:28.361325 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2f17e-a4db-4627-9705-cabdb0937174 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:28.362422 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2f17e-a4db-4627-9705-cabdb0937174 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:28.363075 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2f17e-a4db-4627-9705-cabdb0937174 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:28.366756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-c1b2f17e-a4db-4627-9705-cabdb0937174 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:28.423736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 988/1970] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:28:28.425376 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80089613-2cd0-4cac-9fb3-1e0afbec75b9 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:28.425855 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80089613-2cd0-4cac-9fb3-1e0afbec75b9 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:28.426210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80089613-2cd0-4cac-9fb3-1e0afbec75b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:28.449224 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-80089613-2cd0-4cac-9fb3-1e0afbec75b9 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:28.450676 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-80089613-2cd0-4cac-9fb3-1e0afbec75b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:28.478071 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 989/1971] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:28.482376 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80089613-2cd0-4cac-9fb3-1e0afbec75b9 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:28.494247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80089613-2cd0-4cac-9fb3-1e0afbec75b9 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:28.500544 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107640a0-7916-418b-b2ef-44e62a0cf2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:28.501777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107640a0-7916-418b-b2ef-44e62a0cf2cb admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:28.502123 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107640a0-7916-418b-b2ef-44e62a0cf2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:28.502454 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107640a0-7916-418b-b2ef-44e62a0cf2cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:28.503173 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107640a0-7916-418b-b2ef-44e62a0cf2cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:28:28.504182 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107640a0-7916-418b-b2ef-44e62a0cf2cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:28.505851 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107640a0-7916-418b-b2ef-44e62a0cf2cb admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:28.524250 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b2f17e-a4db-4627-9705-cabdb0937174 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:28.525914 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b2f17e-a4db-4627-9705-cabdb0937174 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:28.528481 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 990/1972] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:28:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:28:28.529435 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107640a0-7916-418b-b2ef-44e62a0cf2cb admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:28.538313 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 983/1973] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:28:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:28:28.540541 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107640a0-7916-418b-b2ef-44e62a0cf2cb admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:28.545444 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c92ac9-02d6-40ab-bf19-cb2bb71bf633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:28.546123 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c92ac9-02d6-40ab-bf19-cb2bb71bf633 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:28.546464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c92ac9-02d6-40ab-bf19-cb2bb71bf633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:28.546820 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c92ac9-02d6-40ab-bf19-cb2bb71bf633 admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:28.547565 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c92ac9-02d6-40ab-bf19-cb2bb71bf633 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:28.549781 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c92ac9-02d6-40ab-bf19-cb2bb71bf633 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:28.588668 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 991/1974] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:28:28 2022] POST /identity/v3/projects => generated 399 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:28:28.591214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2f17e-a4db-4627-9705-cabdb0937174 service neutron] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:28.604218 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2f17e-a4db-4627-9705-cabdb0937174 service neutron] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:28.659232 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:28.689001 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342dff80-c2d4-4586-92ac-d1dd440397c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:28.690190 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342dff80-c2d4-4586-92ac-d1dd440397c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:28.690458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342dff80-c2d4-4586-92ac-d1dd440397c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:28.690850 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342dff80-c2d4-4586-92ac-d1dd440397c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:28.691960 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342dff80-c2d4-4586-92ac-d1dd440397c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:28.694497 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342dff80-c2d4-4586-92ac-d1dd440397c3 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:28.718568 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 984/1975] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:28:28 2022] POST /identity/v3/users => generated 300 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:28:28.723668 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c92ac9-02d6-40ab-bf19-cb2bb71bf633 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:28.734297 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c92ac9-02d6-40ab-bf19-cb2bb71bf633 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:28.738973 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634432d-96a4-4d16-bf45-fc9c8af05338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:28.739597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634432d-96a4-4d16-bf45-fc9c8af05338 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:28.739870 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634432d-96a4-4d16-bf45-fc9c8af05338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:28.740170 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634432d-96a4-4d16-bf45-fc9c8af05338 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:28.740817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634432d-96a4-4d16-bf45-fc9c8af05338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:28.742418 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634432d-96a4-4d16-bf45-fc9c8af05338 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:28.753237 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 992/1976] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:28.760170 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342dff80-c2d4-4586-92ac-d1dd440397c3 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:28.772317 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342dff80-c2d4-4586-92ac-d1dd440397c3 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:28.777025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9382059d-a973-4273-ae85-d2909c5cec7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:28.777833 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382059d-a973-4273-ae85-d2909c5cec7c admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:28.778114 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382059d-a973-4273-ae85-d2909c5cec7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:28.778388 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382059d-a973-4273-ae85-d2909c5cec7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:28.779040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9382059d-a973-4273-ae85-d2909c5cec7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:28.780710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9382059d-a973-4273-ae85-d2909c5cec7c admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:28.791810 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 985/1977] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:28.796034 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634432d-96a4-4d16-bf45-fc9c8af05338 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:28.806594 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634432d-96a4-4d16-bf45-fc9c8af05338 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:28.811263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c0927b-f429-4fb9-ba76-50f7396be366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:28.811950 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0927b-f429-4fb9-ba76-50f7396be366 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:28.812238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0927b-f429-4fb9-ba76-50f7396be366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:28.812516 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0927b-f429-4fb9-ba76-50f7396be366 admin admin] PATH_INFO: `/v3/projects/241d0dbfc72845c09fbb893a9dbce085/users/7a7ea904a35f4692bdc3c566ad6b1c63/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:28.829817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c0927b-f429-4fb9-ba76-50f7396be366 admin admin] RBAC: Authorizing `identity:create_grant(project_id=241d0dbfc72845c09fbb893a9dbce085, user_id=7a7ea904a35f4692bdc3c566ad6b1c63, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:28.832781 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c0927b-f429-4fb9-ba76-50f7396be366 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:28.835154 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:28.849290 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 993/1978] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:28:28 2022] PUT /identity/v3/projects/241d0dbfc72845c09fbb893a9dbce085/users/7a7ea904a35f4692bdc3c566ad6b1c63/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:28.860450 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637865c6-e088-4d50-a7d1-295d0b51b635 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:28.860774 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637865c6-e088-4d50-a7d1-295d0b51b635 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:28.861158 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637865c6-e088-4d50-a7d1-295d0b51b635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:28.890257 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-637865c6-e088-4d50-a7d1-295d0b51b635 None None] MFA Rules not processed for user `7a7ea904a35f4692bdc3c566ad6b1c63`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:28.891482 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-637865c6-e088-4d50-a7d1-295d0b51b635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:28.966774 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 986/1979] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:28 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:28.978588 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c0927b-f429-4fb9-ba76-50f7396be366 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:28.980459 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-78c0927b-f429-4fb9-ba76-50f7396be366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:29.046321 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c0927b-f429-4fb9-ba76-50f7396be366 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:29.050198 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70de32a-bf17-4a8a-a809-94ee4cc6984e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:29.050817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70de32a-bf17-4a8a-a809-94ee4cc6984e service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:29.051175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70de32a-bf17-4a8a-a809-94ee4cc6984e service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:29.051615 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70de32a-bf17-4a8a-a809-94ee4cc6984e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:29.067862 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70de32a-bf17-4a8a-a809-94ee4cc6984e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:29.069474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70de32a-bf17-4a8a-a809-94ee4cc6984e service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:29.081087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 994/1980] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:28:28 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:28:29.584986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637865c6-e088-4d50-a7d1-295d0b51b635 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:29.588716 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-637865c6-e088-4d50-a7d1-295d0b51b635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:29.766359 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637865c6-e088-4d50-a7d1-295d0b51b635 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:29.776765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929c6c8-a772-4991-abf7-c47714e5e0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:29.780318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929c6c8-a772-4991-abf7-c47714e5e0b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:29.781010 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929c6c8-a772-4991-abf7-c47714e5e0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:29.781316 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929c6c8-a772-4991-abf7-c47714e5e0b4 admin admin] PATH_INFO: `/v3/projects/48f4b081193e41cfb1f483fb8d18c045` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:29.785228 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929c6c8-a772-4991-abf7-c47714e5e0b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48f4b081193e41cfb1f483fb8d18c045)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:29.787451 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929c6c8-a772-4991-abf7-c47714e5e0b4 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:29.874963 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-0929c6c8-a772-4991-abf7-c47714e5e0b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48f4b081193e41cfb1f483fb8d18c045', 'request_id': 'req-0929c6c8-a772-4991-abf7-c47714e5e0b4'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:29.877059 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:29.881910 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-0929c6c8-a772-4991-abf7-c47714e5e0b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48f4b081193e41cfb1f483fb8d18c045', 'request_id': 'req-0929c6c8-a772-4991-abf7-c47714e5e0b4'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:29.891141 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-0929c6c8-a772-4991-abf7-c47714e5e0b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48f4b081193e41cfb1f483fb8d18c045', 'request_id': 'req-0929c6c8-a772-4991-abf7-c47714e5e0b4'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:29.902449 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-0929c6c8-a772-4991-abf7-c47714e5e0b4 admin admin] The token cache is being invalidate because project 48f4b081193e41cfb1f483fb8d18c045 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:28:29.902734 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-0929c6c8-a772-4991-abf7-c47714e5e0b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:29.903345 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-0929c6c8-a772-4991-abf7-c47714e5e0b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:29.904886 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 987/1981] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:28:29 2022] DELETE /identity/v3/projects/48f4b081193e41cfb1f483fb8d18c045 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:29.909220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70de32a-bf17-4a8a-a809-94ee4cc6984e service cinder] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:29.919896 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70de32a-bf17-4a8a-a809-94ee4cc6984e service cinder] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:29.989211 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974e9f8-af44-488d-a6c9-a9122ea505c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:29.989835 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974e9f8-af44-488d-a6c9-a9122ea505c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:29.990136 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974e9f8-af44-488d-a6c9-a9122ea505c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:29.990423 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974e9f8-af44-488d-a6c9-a9122ea505c8 admin admin] PATH_INFO: `/v3/users/f1646b8caaa343f5adabd84982f4bad9` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:29.991589 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974e9f8-af44-488d-a6c9-a9122ea505c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1646b8caaa343f5adabd84982f4bad9)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:29.993637 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974e9f8-af44-488d-a6c9-a9122ea505c8 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:30.045630 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-3974e9f8-af44-488d-a6c9-a9122ea505c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1646b8caaa343f5adabd84982f4bad9'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:30.046323 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-3974e9f8-af44-488d-a6c9-a9122ea505c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1646b8caaa343f5adabd84982f4bad9'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:30.055151 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-3974e9f8-af44-488d-a6c9-a9122ea505c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1646b8caaa343f5adabd84982f4bad9'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:30.070014 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-3974e9f8-af44-488d-a6c9-a9122ea505c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1646b8caaa343f5adabd84982f4bad9'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:30.079764 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-3974e9f8-af44-488d-a6c9-a9122ea505c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1646b8caaa343f5adabd84982f4bad9'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:30.086255 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 995/1982] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:28:29 2022] DELETE /identity/v3/users/f1646b8caaa343f5adabd84982f4bad9 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:30.514354 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929c6c8-a772-4991-abf7-c47714e5e0b4 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:30.515874 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-0929c6c8-a772-4991-abf7-c47714e5e0b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:30.531434 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929c6c8-a772-4991-abf7-c47714e5e0b4 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:30.608566 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d4bf13-12c5-4b70-a696-bba2c3df7c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:30.610052 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d4bf13-12c5-4b70-a696-bba2c3df7c6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:30.610409 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d4bf13-12c5-4b70-a696-bba2c3df7c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:30.610751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d4bf13-12c5-4b70-a696-bba2c3df7c6b admin admin] PATH_INFO: `/v3/projects/b7c4b94378ee4023a95540195c38b9e2` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:30.612204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d4bf13-12c5-4b70-a696-bba2c3df7c6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7c4b94378ee4023a95540195c38b9e2)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:30.614192 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d4bf13-12c5-4b70-a696-bba2c3df7c6b admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:30.667525 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-25d4bf13-12c5-4b70-a696-bba2c3df7c6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7c4b94378ee4023a95540195c38b9e2', 'request_id': 'req-25d4bf13-12c5-4b70-a696-bba2c3df7c6b'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:30.674007 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-25d4bf13-12c5-4b70-a696-bba2c3df7c6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7c4b94378ee4023a95540195c38b9e2', 'request_id': 'req-25d4bf13-12c5-4b70-a696-bba2c3df7c6b'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:30.682103 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-25d4bf13-12c5-4b70-a696-bba2c3df7c6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7c4b94378ee4023a95540195c38b9e2', 'request_id': 'req-25d4bf13-12c5-4b70-a696-bba2c3df7c6b'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:30.693031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-25d4bf13-12c5-4b70-a696-bba2c3df7c6b admin admin] The token cache is being invalidate because project b7c4b94378ee4023a95540195c38b9e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:28:30.693317 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-25d4bf13-12c5-4b70-a696-bba2c3df7c6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:30.693943 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-25d4bf13-12c5-4b70-a696-bba2c3df7c6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:30.695420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 988/1983] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:28:30 2022] DELETE /identity/v3/projects/b7c4b94378ee4023a95540195c38b9e2 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:30.700366 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e803d6f-d1b4-43d0-80f9-e2e54192e8c7 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:30.700654 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e803d6f-d1b4-43d0-80f9-e2e54192e8c7 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:30.700930 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e803d6f-d1b4-43d0-80f9-e2e54192e8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:30.721500 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-0e803d6f-d1b4-43d0-80f9-e2e54192e8c7 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:30.723257 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-0e803d6f-d1b4-43d0-80f9-e2e54192e8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:30.735444 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:30.791946 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 996/1984] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:28:30.801267 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fad0e-54f1-4683-83c9-9015fdd05d72 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:30.801558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fad0e-54f1-4683-83c9-9015fdd05d72 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:30.801826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fad0e-54f1-4683-83c9-9015fdd05d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:30.822456 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-276fad0e-54f1-4683-83c9-9015fdd05d72 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:30.823624 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-276fad0e-54f1-4683-83c9-9015fdd05d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:30.832967 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 989/1985] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:30.836926 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e803d6f-d1b4-43d0-80f9-e2e54192e8c7 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:30.853438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e803d6f-d1b4-43d0-80f9-e2e54192e8c7 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:30.860721 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e14af-3c30-4b71-bd90-925129c0ce0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:30.861566 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e14af-3c30-4b71-bd90-925129c0ce0a admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:30.861984 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e14af-3c30-4b71-bd90-925129c0ce0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:30.862995 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e14af-3c30-4b71-bd90-925129c0ce0a admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:30.864002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e14af-3c30-4b71-bd90-925129c0ce0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:28:30.864627 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e14af-3c30-4b71-bd90-925129c0ce0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:30.867434 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e14af-3c30-4b71-bd90-925129c0ce0a admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:30.886056 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 997/1986] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:28:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:28:30.890405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276fad0e-54f1-4683-83c9-9015fdd05d72 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:30.903070 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276fad0e-54f1-4683-83c9-9015fdd05d72 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:30.910154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5663708-24ab-42b4-8915-80b16229161f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:30.910855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5663708-24ab-42b4-8915-80b16229161f admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:30.911714 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5663708-24ab-42b4-8915-80b16229161f admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:30.911714 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5663708-24ab-42b4-8915-80b16229161f admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:30.912544 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5663708-24ab-42b4-8915-80b16229161f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:30.915145 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5663708-24ab-42b4-8915-80b16229161f admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:30.962478 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 990/1987] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:28:30 2022] POST /identity/v3/projects => generated 401 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:28:30.965901 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e14af-3c30-4b71-bd90-925129c0ce0a admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:30.980293 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e14af-3c30-4b71-bd90-925129c0ce0a admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:31.113367 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd98d3-2d01-4933-90db-1f29ca68f333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:31.114728 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd98d3-2d01-4933-90db-1f29ca68f333 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:31.115960 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd98d3-2d01-4933-90db-1f29ca68f333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:31.116420 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd98d3-2d01-4933-90db-1f29ca68f333 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:31.117555 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afd98d3-2d01-4933-90db-1f29ca68f333 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:31.120401 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afd98d3-2d01-4933-90db-1f29ca68f333 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:31.147016 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 998/1988] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:28:30 2022] POST /identity/v3/users => generated 301 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:28:31.151326 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5663708-24ab-42b4-8915-80b16229161f admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:31.162071 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5663708-24ab-42b4-8915-80b16229161f admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:31.166639 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57005475-6a59-4a68-8eda-b18c558456ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:31.167250 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57005475-6a59-4a68-8eda-b18c558456ac admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:31.167524 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57005475-6a59-4a68-8eda-b18c558456ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:31.167813 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57005475-6a59-4a68-8eda-b18c558456ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:31.168497 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57005475-6a59-4a68-8eda-b18c558456ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:31.170165 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57005475-6a59-4a68-8eda-b18c558456ac admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:31.180549 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 991/1989] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:31.184733 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd98d3-2d01-4933-90db-1f29ca68f333 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:31.194994 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd98d3-2d01-4933-90db-1f29ca68f333 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:31.199602 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de333425-78ca-4f89-a516-bba1bf49dbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:31.200234 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de333425-78ca-4f89-a516-bba1bf49dbf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:31.200512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de333425-78ca-4f89-a516-bba1bf49dbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:31.200784 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de333425-78ca-4f89-a516-bba1bf49dbf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:31.201426 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de333425-78ca-4f89-a516-bba1bf49dbf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:31.203005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de333425-78ca-4f89-a516-bba1bf49dbf9 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:31.214328 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 999/1990] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:31.220501 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57005475-6a59-4a68-8eda-b18c558456ac admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:31.231142 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57005475-6a59-4a68-8eda-b18c558456ac admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:31.236004 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c14980-75cf-47e9-90b1-b553a07d732c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:31.236677 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c14980-75cf-47e9-90b1-b553a07d732c admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:31.236955 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c14980-75cf-47e9-90b1-b553a07d732c admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:31.237232 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c14980-75cf-47e9-90b1-b553a07d732c admin admin] PATH_INFO: `/v3/projects/e013251a4c4446d5b51d7682fe97c4ca/users/5e8039738afa4c24829979a67adf487e/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:31.252930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c14980-75cf-47e9-90b1-b553a07d732c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e013251a4c4446d5b51d7682fe97c4ca, user_id=5e8039738afa4c24829979a67adf487e, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:31.255708 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c14980-75cf-47e9-90b1-b553a07d732c admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:31.269710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 992/1991] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:28:31 2022] PUT /identity/v3/projects/e013251a4c4446d5b51d7682fe97c4ca/users/5e8039738afa4c24829979a67adf487e/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:31.280516 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3f469-d02a-4f8c-b343-c7bef881bb60 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:31.280810 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3f469-d02a-4f8c-b343-c7bef881bb60 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:31.281089 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3f469-d02a-4f8c-b343-c7bef881bb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:31.311125 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-d5d3f469-d02a-4f8c-b343-c7bef881bb60 None None] MFA Rules not processed for user `5e8039738afa4c24829979a67adf487e`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:31.312544 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-d5d3f469-d02a-4f8c-b343-c7bef881bb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:31.378238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1000/1992] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:31 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:31.382766 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c14980-75cf-47e9-90b1-b553a07d732c admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:31.391942 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:31.450961 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c14980-75cf-47e9-90b1-b553a07d732c admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:31.456795 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c017031-b83e-4d19-b0b3-d288b9c892aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:31.457962 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c017031-b83e-4d19-b0b3-d288b9c892aa admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:31.457962 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c017031-b83e-4d19-b0b3-d288b9c892aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:31.458178 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c017031-b83e-4d19-b0b3-d288b9c892aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:31.458719 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c017031-b83e-4d19-b0b3-d288b9c892aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:31.460692 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c017031-b83e-4d19-b0b3-d288b9c892aa admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:31.496206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 993/1993] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:28:31 2022] POST /identity/v3/projects => generated 399 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:28:31.497390 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3f469-d02a-4f8c-b343-c7bef881bb60 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:31.509176 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3f469-d02a-4f8c-b343-c7bef881bb60 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:31.518312 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:31.572915 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25863e2d-4602-4b37-a089-f370c7244c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:31.573510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25863e2d-4602-4b37-a089-f370c7244c71 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:31.573791 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25863e2d-4602-4b37-a089-f370c7244c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:31.574059 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25863e2d-4602-4b37-a089-f370c7244c71 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:31.574765 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25863e2d-4602-4b37-a089-f370c7244c71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:31.576769 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25863e2d-4602-4b37-a089-f370c7244c71 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:31.601697 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1001/1994] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:28:31 2022] POST /identity/v3/users => generated 306 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:28:31.605671 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c017031-b83e-4d19-b0b3-d288b9c892aa admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:31.617220 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c017031-b83e-4d19-b0b3-d288b9c892aa admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:31.624355 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5503b7e-de01-4ca6-ae0e-641220214f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:31.625050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5503b7e-de01-4ca6-ae0e-641220214f93 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:31.625338 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5503b7e-de01-4ca6-ae0e-641220214f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:31.625623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5503b7e-de01-4ca6-ae0e-641220214f93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:31.626373 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5503b7e-de01-4ca6-ae0e-641220214f93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:31.628115 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5503b7e-de01-4ca6-ae0e-641220214f93 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:31.641718 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 994/1995] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:31.645772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25863e2d-4602-4b37-a089-f370c7244c71 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:31.656843 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25863e2d-4602-4b37-a089-f370c7244c71 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:31.661598 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7831045-bb83-4423-993c-e67248c7ee24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:31.662274 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7831045-bb83-4423-993c-e67248c7ee24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:31.662559 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7831045-bb83-4423-993c-e67248c7ee24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:31.662886 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7831045-bb83-4423-993c-e67248c7ee24 admin admin] PATH_INFO: `/v3/projects/1705f0867e444fe7b077791f13650e72/users/ae5d7c09db1e490eb86bb33a33aa48b8/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:31.678858 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7831045-bb83-4423-993c-e67248c7ee24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1705f0867e444fe7b077791f13650e72, user_id=ae5d7c09db1e490eb86bb33a33aa48b8, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:31.681567 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7831045-bb83-4423-993c-e67248c7ee24 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:31.697088 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1002/1996] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:28:31 2022] PUT /identity/v3/projects/1705f0867e444fe7b077791f13650e72/users/ae5d7c09db1e490eb86bb33a33aa48b8/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:31.700962 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5503b7e-de01-4ca6-ae0e-641220214f93 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:31.711103 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5503b7e-de01-4ca6-ae0e-641220214f93 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:31.780447 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3abcf-7fe3-4f8f-9a6b-4968af5291de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:31.781732 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3abcf-7fe3-4f8f-9a6b-4968af5291de admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:31.782054 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3abcf-7fe3-4f8f-9a6b-4968af5291de admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:31.782322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3abcf-7fe3-4f8f-9a6b-4968af5291de admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:31.783645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f3abcf-7fe3-4f8f-9a6b-4968af5291de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:31.784840 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f3abcf-7fe3-4f8f-9a6b-4968af5291de admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:31.797231 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 995/1997] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:31.802355 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7831045-bb83-4423-993c-e67248c7ee24 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:31.815050 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7831045-bb83-4423-993c-e67248c7ee24 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:31.820865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d33027-0fd2-487f-b879-25ca69352113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:31.821723 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d33027-0fd2-487f-b879-25ca69352113 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:31.822007 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d33027-0fd2-487f-b879-25ca69352113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:31.822280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d33027-0fd2-487f-b879-25ca69352113 admin admin] PATH_INFO: `/v3/projects/1705f0867e444fe7b077791f13650e72/users/ae5d7c09db1e490eb86bb33a33aa48b8/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:31.825306 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d33027-0fd2-487f-b879-25ca69352113 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1705f0867e444fe7b077791f13650e72, user_id=ae5d7c09db1e490eb86bb33a33aa48b8, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:31.828865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d33027-0fd2-487f-b879-25ca69352113 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:31.840483 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1003/1998] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:28:31 2022] PUT /identity/v3/projects/1705f0867e444fe7b077791f13650e72/users/ae5d7c09db1e490eb86bb33a33aa48b8/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:31.851536 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b355df4-049b-452a-8c10-4e9975d0d099 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:31.851938 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b355df4-049b-452a-8c10-4e9975d0d099 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:31.852281 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b355df4-049b-452a-8c10-4e9975d0d099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:31.883533 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-1b355df4-049b-452a-8c10-4e9975d0d099 None None] MFA Rules not processed for user `ae5d7c09db1e490eb86bb33a33aa48b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:31.884889 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-1b355df4-049b-452a-8c10-4e9975d0d099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:31.968355 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 996/1999] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:31 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:31.976523 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d33027-0fd2-487f-b879-25ca69352113 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:31.978590 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-32d33027-0fd2-487f-b879-25ca69352113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:32.047830 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d33027-0fd2-487f-b879-25ca69352113 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:32.051610 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c167a0-b6e4-44d1-8385-3ad8777e4061 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:32.052357 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c167a0-b6e4-44d1-8385-3ad8777e4061 service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:32.052694 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c167a0-b6e4-44d1-8385-3ad8777e4061 service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:32.053023 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c167a0-b6e4-44d1-8385-3ad8777e4061 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:32.062686 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c167a0-b6e4-44d1-8385-3ad8777e4061 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:32.064675 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c167a0-b6e4-44d1-8385-3ad8777e4061 service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:32.075192 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1004/2000] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:28:31 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:28:32.163508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b355df4-049b-452a-8c10-4e9975d0d099 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:32.165465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-1b355df4-049b-452a-8c10-4e9975d0d099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:32.174273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:32.182661 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b355df4-049b-452a-8c10-4e9975d0d099 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:32.187409 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40971309-adc6-45c2-be11-58d80c30ec24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:32.188387 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40971309-adc6-45c2-be11-58d80c30ec24 service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:32.188742 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40971309-adc6-45c2-be11-58d80c30ec24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:32.189106 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40971309-adc6-45c2-be11-58d80c30ec24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:32.249251 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40971309-adc6-45c2-be11-58d80c30ec24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:32.250809 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40971309-adc6-45c2-be11-58d80c30ec24 service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:32.260808 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 997/2001] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:28:32 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:28:37.488693 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c167a0-b6e4-44d1-8385-3ad8777e4061 service cinder] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:37.550012 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c167a0-b6e4-44d1-8385-3ad8777e4061 service cinder] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:37.554737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eecd94-ffe3-45b2-a1f7-2c93910f0932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:37.555366 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eecd94-ffe3-45b2-a1f7-2c93910f0932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:37.555638 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eecd94-ffe3-45b2-a1f7-2c93910f0932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:37.555877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eecd94-ffe3-45b2-a1f7-2c93910f0932 admin admin] PATH_INFO: `/v3/users/5e8039738afa4c24829979a67adf487e` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:37.557058 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eecd94-ffe3-45b2-a1f7-2c93910f0932 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e8039738afa4c24829979a67adf487e)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:37.558987 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eecd94-ffe3-45b2-a1f7-2c93910f0932 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:37.567259 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:37.568450 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40971309-adc6-45c2-be11-58d80c30ec24 service cinder] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:37.570128 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-40971309-adc6-45c2-be11-58d80c30ec24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:37.585564 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40971309-adc6-45c2-be11-58d80c30ec24 service cinder] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:37.591693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6b548-e0e3-44d2-81ec-14a4ffb4f4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:37.592566 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6b548-e0e3-44d2-81ec-14a4ffb4f4a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:37.593025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6b548-e0e3-44d2-81ec-14a4ffb4f4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:37.593422 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6b548-e0e3-44d2-81ec-14a4ffb4f4a3 admin admin] PATH_INFO: `/v3/users/7a7ea904a35f4692bdc3c566ad6b1c63` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:37.594909 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec6b548-e0e3-44d2-81ec-14a4ffb4f4a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a7ea904a35f4692bdc3c566ad6b1c63)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:37.597706 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec6b548-e0e3-44d2-81ec-14a4ffb4f4a3 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:37.610609 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-12eecd94-ffe3-45b2-a1f7-2c93910f0932 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e8039738afa4c24829979a67adf487e'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:37.611264 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-12eecd94-ffe3-45b2-a1f7-2c93910f0932 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e8039738afa4c24829979a67adf487e'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:37.619422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-12eecd94-ffe3-45b2-a1f7-2c93910f0932 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e8039738afa4c24829979a67adf487e'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:37.633683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-12eecd94-ffe3-45b2-a1f7-2c93910f0932 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e8039738afa4c24829979a67adf487e'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:37.641351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-12eecd94-ffe3-45b2-a1f7-2c93910f0932 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e8039738afa4c24829979a67adf487e'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:37.646457 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-0ec6b548-e0e3-44d2-81ec-14a4ffb4f4a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a7ea904a35f4692bdc3c566ad6b1c63'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:37.648605 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1005/2002] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:28:37 2022] DELETE /identity/v3/users/5e8039738afa4c24829979a67adf487e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:37.653310 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86d43c-7ff6-4bdc-b81a-2db34709023c None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:37.653606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86d43c-7ff6-4bdc-b81a-2db34709023c None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:37.653883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86d43c-7ff6-4bdc-b81a-2db34709023c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:37.654803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-0ec6b548-e0e3-44d2-81ec-14a4ffb4f4a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a7ea904a35f4692bdc3c566ad6b1c63'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:37.668425 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-0ec6b548-e0e3-44d2-81ec-14a4ffb4f4a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a7ea904a35f4692bdc3c566ad6b1c63'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:37.669162 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-0ec6b548-e0e3-44d2-81ec-14a4ffb4f4a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a7ea904a35f4692bdc3c566ad6b1c63'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:37.674952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-9d86d43c-7ff6-4bdc-b81a-2db34709023c None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:37.676279 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-9d86d43c-7ff6-4bdc-b81a-2db34709023c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:37.676971 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-0ec6b548-e0e3-44d2-81ec-14a4ffb4f4a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a7ea904a35f4692bdc3c566ad6b1c63'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:37.683047 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 998/2003] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:28:37 2022] DELETE /identity/v3/users/7a7ea904a35f4692bdc3c566ad6b1c63 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:37.687315 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e72bf0-9f8b-45b2-8e95-4f060e2292ad None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:37.687597 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e72bf0-9f8b-45b2-8e95-4f060e2292ad None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:37.687867 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e72bf0-9f8b-45b2-8e95-4f060e2292ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:37.708184 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-c3e72bf0-9f8b-45b2-8e95-4f060e2292ad None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:37.709377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-c3e72bf0-9f8b-45b2-8e95-4f060e2292ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:37.740657 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1006/2004] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:28:37.748098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86d43c-7ff6-4bdc-b81a-2db34709023c None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:37.749669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-9d86d43c-7ff6-4bdc-b81a-2db34709023c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:37.775947 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 999/2005] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:28:37.788066 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e72bf0-9f8b-45b2-8e95-4f060e2292ad None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:37.789945 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-c3e72bf0-9f8b-45b2-8e95-4f060e2292ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:37.807034 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:37.838644 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86d43c-7ff6-4bdc-b81a-2db34709023c None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:37.870782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e72bf0-9f8b-45b2-8e95-4f060e2292ad None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:37.881193 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6105e1-7d6d-4ce1-b4c0-695d8e2f1b37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:37.883035 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6105e1-7d6d-4ce1-b4c0-695d8e2f1b37 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:37.883499 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6105e1-7d6d-4ce1-b4c0-695d8e2f1b37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:37.883998 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6105e1-7d6d-4ce1-b4c0-695d8e2f1b37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:37.897362 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6105e1-7d6d-4ce1-b4c0-695d8e2f1b37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:37.899503 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6105e1-7d6d-4ce1-b4c0-695d8e2f1b37 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:37.901812 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:37.914587 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1000/2006] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:28:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:28:37.921466 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e70d3-0e44-473b-900c-55399eab3957 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:37.922220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e70d3-0e44-473b-900c-55399eab3957 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:37.922735 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e70d3-0e44-473b-900c-55399eab3957 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:37.923060 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e70d3-0e44-473b-900c-55399eab3957 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:37.933838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e70d3-0e44-473b-900c-55399eab3957 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:37.935850 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e70d3-0e44-473b-900c-55399eab3957 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:37.947304 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1007/2007] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:28:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:28:38.226971 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6105e1-7d6d-4ce1-b4c0-695d8e2f1b37 service neutron] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:38.229144 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-4e6105e1-7d6d-4ce1-b4c0-695d8e2f1b37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:38.248317 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6105e1-7d6d-4ce1-b4c0-695d8e2f1b37 service neutron] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:38.255085 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035c4d0-c009-49b7-a26c-440843db6ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:38.256696 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035c4d0-c009-49b7-a26c-440843db6ad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:38.257143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035c4d0-c009-49b7-a26c-440843db6ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:38.257591 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035c4d0-c009-49b7-a26c-440843db6ad4 admin admin] PATH_INFO: `/v3/projects/241d0dbfc72845c09fbb893a9dbce085` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:38.259093 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a035c4d0-c009-49b7-a26c-440843db6ad4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=241d0dbfc72845c09fbb893a9dbce085)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:38.261186 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a035c4d0-c009-49b7-a26c-440843db6ad4 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:38.314727 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a035c4d0-c009-49b7-a26c-440843db6ad4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '241d0dbfc72845c09fbb893a9dbce085', 'request_id': 'req-a035c4d0-c009-49b7-a26c-440843db6ad4'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:38.322161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a035c4d0-c009-49b7-a26c-440843db6ad4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '241d0dbfc72845c09fbb893a9dbce085', 'request_id': 'req-a035c4d0-c009-49b7-a26c-440843db6ad4'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:38.332364 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a035c4d0-c009-49b7-a26c-440843db6ad4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '241d0dbfc72845c09fbb893a9dbce085', 'request_id': 'req-a035c4d0-c009-49b7-a26c-440843db6ad4'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:38.348228 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a035c4d0-c009-49b7-a26c-440843db6ad4 admin admin] The token cache is being invalidate because project 241d0dbfc72845c09fbb893a9dbce085 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:28:38.348530 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a035c4d0-c009-49b7-a26c-440843db6ad4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:38.349179 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a035c4d0-c009-49b7-a26c-440843db6ad4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:38.350870 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1001/2008] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:28:38 2022] DELETE /identity/v3/projects/241d0dbfc72845c09fbb893a9dbce085 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:38.543095 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e70d3-0e44-473b-900c-55399eab3957 service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:38.545195 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-084e70d3-0e44-473b-900c-55399eab3957 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:38.580502 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e70d3-0e44-473b-900c-55399eab3957 service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:38.643214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad602830-7889-4836-af8b-576403fc8611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:38.643872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad602830-7889-4836-af8b-576403fc8611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:38.644245 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad602830-7889-4836-af8b-576403fc8611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:38.644583 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad602830-7889-4836-af8b-576403fc8611 admin admin] PATH_INFO: `/v3/projects/e013251a4c4446d5b51d7682fe97c4ca` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:38.645862 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad602830-7889-4836-af8b-576403fc8611 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e013251a4c4446d5b51d7682fe97c4ca)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:38.648886 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad602830-7889-4836-af8b-576403fc8611 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:38.726220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ad602830-7889-4836-af8b-576403fc8611 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e013251a4c4446d5b51d7682fe97c4ca', 'request_id': 'req-ad602830-7889-4836-af8b-576403fc8611'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:38.733324 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ad602830-7889-4836-af8b-576403fc8611 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e013251a4c4446d5b51d7682fe97c4ca', 'request_id': 'req-ad602830-7889-4836-af8b-576403fc8611'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:38.743236 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ad602830-7889-4836-af8b-576403fc8611 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e013251a4c4446d5b51d7682fe97c4ca', 'request_id': 'req-ad602830-7889-4836-af8b-576403fc8611'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:38.770645 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ad602830-7889-4836-af8b-576403fc8611 admin admin] The token cache is being invalidate because project e013251a4c4446d5b51d7682fe97c4ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:28:38.771239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ad602830-7889-4836-af8b-576403fc8611 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:38.772352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ad602830-7889-4836-af8b-576403fc8611 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:38.775258 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1008/2009] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:28:38 2022] DELETE /identity/v3/projects/e013251a4c4446d5b51d7682fe97c4ca => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:38.781817 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035c4d0-c009-49b7-a26c-440843db6ad4 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:38.784501 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-a035c4d0-c009-49b7-a26c-440843db6ad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:38.799726 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035c4d0-c009-49b7-a26c-440843db6ad4 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:38.879320 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fea1cc-ef89-46d2-bf33-c1d000dc8719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:38.880071 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fea1cc-ef89-46d2-bf33-c1d000dc8719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:38.880351 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fea1cc-ef89-46d2-bf33-c1d000dc8719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:38.880627 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fea1cc-ef89-46d2-bf33-c1d000dc8719 admin admin] PATH_INFO: `/v3/users/ae5d7c09db1e490eb86bb33a33aa48b8` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:38.881959 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fea1cc-ef89-46d2-bf33-c1d000dc8719 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae5d7c09db1e490eb86bb33a33aa48b8)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:38.884159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fea1cc-ef89-46d2-bf33-c1d000dc8719 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:38.930571 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:38.936654 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-40fea1cc-ef89-46d2-bf33-c1d000dc8719 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae5d7c09db1e490eb86bb33a33aa48b8'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:38.944983 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-40fea1cc-ef89-46d2-bf33-c1d000dc8719 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae5d7c09db1e490eb86bb33a33aa48b8'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:38.957821 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-40fea1cc-ef89-46d2-bf33-c1d000dc8719 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae5d7c09db1e490eb86bb33a33aa48b8'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:38.958474 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-40fea1cc-ef89-46d2-bf33-c1d000dc8719 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae5d7c09db1e490eb86bb33a33aa48b8'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:38.965254 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-40fea1cc-ef89-46d2-bf33-c1d000dc8719 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae5d7c09db1e490eb86bb33a33aa48b8'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:38.971221 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1002/2010] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:28:38 2022] DELETE /identity/v3/users/ae5d7c09db1e490eb86bb33a33aa48b8 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:39.246255 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad602830-7889-4836-af8b-576403fc8611 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:39.249595 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-ad602830-7889-4836-af8b-576403fc8611 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:39.271622 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad602830-7889-4836-af8b-576403fc8611 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:39.331947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01a91d-99f5-4ef0-9d67-8637b9ed1337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:39.332580 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01a91d-99f5-4ef0-9d67-8637b9ed1337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:39.332861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01a91d-99f5-4ef0-9d67-8637b9ed1337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:39.333129 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01a91d-99f5-4ef0-9d67-8637b9ed1337 admin admin] PATH_INFO: `/v3/projects/1705f0867e444fe7b077791f13650e72` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:39.334305 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e01a91d-99f5-4ef0-9d67-8637b9ed1337 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1705f0867e444fe7b077791f13650e72)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:39.336400 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e01a91d-99f5-4ef0-9d67-8637b9ed1337 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:39.356761 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:39.390529 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8e01a91d-99f5-4ef0-9d67-8637b9ed1337 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1705f0867e444fe7b077791f13650e72', 'request_id': 'req-8e01a91d-99f5-4ef0-9d67-8637b9ed1337'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:39.397271 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8e01a91d-99f5-4ef0-9d67-8637b9ed1337 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1705f0867e444fe7b077791f13650e72', 'request_id': 'req-8e01a91d-99f5-4ef0-9d67-8637b9ed1337'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:39.405377 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8e01a91d-99f5-4ef0-9d67-8637b9ed1337 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1705f0867e444fe7b077791f13650e72', 'request_id': 'req-8e01a91d-99f5-4ef0-9d67-8637b9ed1337'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:39.416231 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8e01a91d-99f5-4ef0-9d67-8637b9ed1337 admin admin] The token cache is being invalidate because project 1705f0867e444fe7b077791f13650e72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:28:39.416515 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8e01a91d-99f5-4ef0-9d67-8637b9ed1337 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:39.417170 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8e01a91d-99f5-4ef0-9d67-8637b9ed1337 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:39.418773 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1009/2011] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:28:39 2022] DELETE /identity/v3/projects/1705f0867e444fe7b077791f13650e72 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:39.424366 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250238aa-94a4-44db-b5f7-af0e601d87f8 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:39.424656 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250238aa-94a4-44db-b5f7-af0e601d87f8 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:39.424957 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250238aa-94a4-44db-b5f7-af0e601d87f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:39.446698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-250238aa-94a4-44db-b5f7-af0e601d87f8 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:39.447934 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-250238aa-94a4-44db-b5f7-af0e601d87f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:39.516316 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1003/2012] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:39.525007 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf09d5-76b5-4bd8-9cde-a81c4ef09fb7 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:39.525287 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf09d5-76b5-4bd8-9cde-a81c4ef09fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:39.525557 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf09d5-76b5-4bd8-9cde-a81c4ef09fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:39.546055 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-b0cf09d5-76b5-4bd8-9cde-a81c4ef09fb7 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:39.547261 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-b0cf09d5-76b5-4bd8-9cde-a81c4ef09fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:39.556136 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1010/2013] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:39.560516 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250238aa-94a4-44db-b5f7-af0e601d87f8 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:39.570591 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250238aa-94a4-44db-b5f7-af0e601d87f8 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:39.575231 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48b593-2b3f-4f2a-8bbc-9ec582f42e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:39.575841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48b593-2b3f-4f2a-8bbc-9ec582f42e4c admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:39.576146 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48b593-2b3f-4f2a-8bbc-9ec582f42e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:39.576412 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48b593-2b3f-4f2a-8bbc-9ec582f42e4c admin admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:39.577055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48b593-2b3f-4f2a-8bbc-9ec582f42e4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:28:39.577353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48b593-2b3f-4f2a-8bbc-9ec582f42e4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:39.579235 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48b593-2b3f-4f2a-8bbc-9ec582f42e4c admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:39.590988 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1004/2014] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:28:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:28:39.595428 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf09d5-76b5-4bd8-9cde-a81c4ef09fb7 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:39.605404 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf09d5-76b5-4bd8-9cde-a81c4ef09fb7 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:39.610571 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fade007-c7c7-4c76-a5cd-4809e562a8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:39.611187 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fade007-c7c7-4c76-a5cd-4809e562a8de admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:39.611464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fade007-c7c7-4c76-a5cd-4809e562a8de admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:39.611845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fade007-c7c7-4c76-a5cd-4809e562a8de admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:39.612516 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fade007-c7c7-4c76-a5cd-4809e562a8de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:39.614423 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fade007-c7c7-4c76-a5cd-4809e562a8de admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:39.642956 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1011/2015] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:28:39 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:28:39.647390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48b593-2b3f-4f2a-8bbc-9ec582f42e4c admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:39.657897 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48b593-2b3f-4f2a-8bbc-9ec582f42e4c admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:39.717939 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff80a6-038d-44e7-83c2-3a449f9b3535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:39.718473 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff80a6-038d-44e7-83c2-3a449f9b3535 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:39.718755 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff80a6-038d-44e7-83c2-3a449f9b3535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:39.719030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff80a6-038d-44e7-83c2-3a449f9b3535 admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:39.719940 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff80a6-038d-44e7-83c2-3a449f9b3535 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:39.721925 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff80a6-038d-44e7-83c2-3a449f9b3535 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:39.745560 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1005/2016] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:28:39 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:28:39.749899 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fade007-c7c7-4c76-a5cd-4809e562a8de admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:39.760539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fade007-c7c7-4c76-a5cd-4809e562a8de admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:39.765404 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f0d49-01d9-4008-8b2e-0d49abeb39d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:39.766145 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f0d49-01d9-4008-8b2e-0d49abeb39d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:39.766627 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f0d49-01d9-4008-8b2e-0d49abeb39d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:39.767081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f0d49-01d9-4008-8b2e-0d49abeb39d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:39.768061 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6f0d49-01d9-4008-8b2e-0d49abeb39d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:39.769955 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6f0d49-01d9-4008-8b2e-0d49abeb39d5 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:39.780537 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1012/2017] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:39.785306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff80a6-038d-44e7-83c2-3a449f9b3535 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:39.795410 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff80a6-038d-44e7-83c2-3a449f9b3535 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:39.800726 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653d5dd-ea1d-4520-b16c-95e1cd9ed648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:39.801319 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653d5dd-ea1d-4520-b16c-95e1cd9ed648 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:39.801597 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653d5dd-ea1d-4520-b16c-95e1cd9ed648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:39.801862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653d5dd-ea1d-4520-b16c-95e1cd9ed648 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:39.802501 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7653d5dd-ea1d-4520-b16c-95e1cd9ed648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:39.804114 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7653d5dd-ea1d-4520-b16c-95e1cd9ed648 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:39.814872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1006/2018] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:39.820837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f0d49-01d9-4008-8b2e-0d49abeb39d5 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:39.831399 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f0d49-01d9-4008-8b2e-0d49abeb39d5 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:39.836719 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d190a6-a80f-4f9c-af0b-255eda161b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:39.837359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d190a6-a80f-4f9c-af0b-255eda161b3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:39.837661 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d190a6-a80f-4f9c-af0b-255eda161b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:39.837934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d190a6-a80f-4f9c-af0b-255eda161b3b admin admin] PATH_INFO: `/v3/projects/48e2b00d845442298d89458d4c29c0d2/users/65b94d2d675f4b30b907579b9a6b175a/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:39.853110 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d190a6-a80f-4f9c-af0b-255eda161b3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=48e2b00d845442298d89458d4c29c0d2, user_id=65b94d2d675f4b30b907579b9a6b175a, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:39.855843 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d190a6-a80f-4f9c-af0b-255eda161b3b admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:39.871085 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1013/2019] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:28:39 2022] PUT /identity/v3/projects/48e2b00d845442298d89458d4c29c0d2/users/65b94d2d675f4b30b907579b9a6b175a/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:39.881984 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a2db4-eebf-4602-beb1-e979900f2eb8 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:39.882311 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a2db4-eebf-4602-beb1-e979900f2eb8 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:39.882713 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a2db4-eebf-4602-beb1-e979900f2eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:39.912445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-0b5a2db4-eebf-4602-beb1-e979900f2eb8 None None] MFA Rules not processed for user `65b94d2d675f4b30b907579b9a6b175a`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:39.913618 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-0b5a2db4-eebf-4602-beb1-e979900f2eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:39.926504 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:39.985386 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1007/2020] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:39.989661 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d190a6-a80f-4f9c-af0b-255eda161b3b admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:40.055670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d190a6-a80f-4f9c-af0b-255eda161b3b admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:40.060497 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185698c6-6d6d-4be5-b0d5-3b189aae232e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:40.061109 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185698c6-6d6d-4be5-b0d5-3b189aae232e admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:40.061386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185698c6-6d6d-4be5-b0d5-3b189aae232e admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:40.061651 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185698c6-6d6d-4be5-b0d5-3b189aae232e admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:40.062272 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185698c6-6d6d-4be5-b0d5-3b189aae232e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:40.064273 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185698c6-6d6d-4be5-b0d5-3b189aae232e admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:40.093662 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1014/2021] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:28:39 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:28:40.098198 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a2db4-eebf-4602-beb1-e979900f2eb8 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:40.109105 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a2db4-eebf-4602-beb1-e979900f2eb8 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:40.168555 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2593faad-c0a3-43a7-99c5-a78b87836486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:40.169166 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2593faad-c0a3-43a7-99c5-a78b87836486 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:40.169442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2593faad-c0a3-43a7-99c5-a78b87836486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:40.169706 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2593faad-c0a3-43a7-99c5-a78b87836486 admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:40.170347 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2593faad-c0a3-43a7-99c5-a78b87836486 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:40.172292 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2593faad-c0a3-43a7-99c5-a78b87836486 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:40.200808 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1008/2022] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:28:40 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:28:40.202872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185698c6-6d6d-4be5-b0d5-3b189aae232e admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:40.217368 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185698c6-6d6d-4be5-b0d5-3b189aae232e admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:40.224362 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d837efb-fad0-474a-8f36-116efcdeeb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:40.224982 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d837efb-fad0-474a-8f36-116efcdeeb29 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:40.225254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d837efb-fad0-474a-8f36-116efcdeeb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:40.225527 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d837efb-fad0-474a-8f36-116efcdeeb29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:40.226183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d837efb-fad0-474a-8f36-116efcdeeb29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:40.227944 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d837efb-fad0-474a-8f36-116efcdeeb29 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:40.240246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1015/2023] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:40.244828 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2593faad-c0a3-43a7-99c5-a78b87836486 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:40.256225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2593faad-c0a3-43a7-99c5-a78b87836486 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:40.261588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e3251d-26c5-4496-a727-dc7336a93da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:40.262246 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e3251d-26c5-4496-a727-dc7336a93da4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:40.262520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e3251d-26c5-4496-a727-dc7336a93da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:40.262824 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e3251d-26c5-4496-a727-dc7336a93da4 admin admin] PATH_INFO: `/v3/projects/da1534be16584d9b88c5e3d62736b273/users/9aae900208d04f798e87b5da0feab672/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:40.278366 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e3251d-26c5-4496-a727-dc7336a93da4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da1534be16584d9b88c5e3d62736b273, user_id=9aae900208d04f798e87b5da0feab672, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:40.281108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e3251d-26c5-4496-a727-dc7336a93da4 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:40.295823 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1009/2024] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:28:40 2022] PUT /identity/v3/projects/da1534be16584d9b88c5e3d62736b273/users/9aae900208d04f798e87b5da0feab672/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:40.299867 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d837efb-fad0-474a-8f36-116efcdeeb29 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:40.310340 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d837efb-fad0-474a-8f36-116efcdeeb29 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:40.325004 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:40.376038 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1cb727-be79-458e-bc03-05a62e1a6fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:40.376653 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cb727-be79-458e-bc03-05a62e1a6fe4 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:40.376930 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cb727-be79-458e-bc03-05a62e1a6fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:40.377200 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cb727-be79-458e-bc03-05a62e1a6fe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:40.377867 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1cb727-be79-458e-bc03-05a62e1a6fe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:40.379441 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1cb727-be79-458e-bc03-05a62e1a6fe4 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:40.390456 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1016/2025] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:40.394472 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e3251d-26c5-4496-a727-dc7336a93da4 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:40.405602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e3251d-26c5-4496-a727-dc7336a93da4 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:40.411243 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64064e16-6c63-48dc-9a24-499820f3b41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:40.411983 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64064e16-6c63-48dc-9a24-499820f3b41f admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:40.412268 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64064e16-6c63-48dc-9a24-499820f3b41f admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:40.412544 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64064e16-6c63-48dc-9a24-499820f3b41f admin admin] PATH_INFO: `/v3/projects/da1534be16584d9b88c5e3d62736b273/users/9aae900208d04f798e87b5da0feab672/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:40.414513 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64064e16-6c63-48dc-9a24-499820f3b41f admin admin] RBAC: Authorizing `identity:create_grant(project_id=da1534be16584d9b88c5e3d62736b273, user_id=9aae900208d04f798e87b5da0feab672, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:40.417408 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64064e16-6c63-48dc-9a24-499820f3b41f admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:40.429055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1010/2026] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:28:40 2022] PUT /identity/v3/projects/da1534be16584d9b88c5e3d62736b273/users/9aae900208d04f798e87b5da0feab672/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:40.440635 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ddbc0-9a19-487c-a4ef-671f4c404dd8 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:40.440931 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ddbc0-9a19-487c-a4ef-671f4c404dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:40.441210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ddbc0-9a19-487c-a4ef-671f4c404dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:40.472762 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-680ddbc0-9a19-487c-a4ef-671f4c404dd8 None None] MFA Rules not processed for user `9aae900208d04f798e87b5da0feab672`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:40.473984 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-680ddbc0-9a19-487c-a4ef-671f4c404dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:40.543098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1017/2027] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:40 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:40.554235 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64064e16-6c63-48dc-9a24-499820f3b41f admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:40.555970 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-64064e16-6c63-48dc-9a24-499820f3b41f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:40.615683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64064e16-6c63-48dc-9a24-499820f3b41f admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:40.620168 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb1e17-3df0-4047-aea2-4af385649a19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:40.620794 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb1e17-3df0-4047-aea2-4af385649a19 service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:40.621077 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb1e17-3df0-4047-aea2-4af385649a19 service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:40.621355 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb1e17-3df0-4047-aea2-4af385649a19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:40.631505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb1e17-3df0-4047-aea2-4af385649a19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:40.633045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb1e17-3df0-4047-aea2-4af385649a19 service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:40.644697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1011/2028] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:28:40 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:28:41.039684 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ddbc0-9a19-487c-a4ef-671f4c404dd8 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:41.107173 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ddbc0-9a19-487c-a4ef-671f4c404dd8 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:41.112439 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b45f17-1413-4d94-94c8-bac0f61f5721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:41.113107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b45f17-1413-4d94-94c8-bac0f61f5721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:41.113397 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b45f17-1413-4d94-94c8-bac0f61f5721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:41.113666 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b45f17-1413-4d94-94c8-bac0f61f5721 admin admin] PATH_INFO: `/v3/users/65b94d2d675f4b30b907579b9a6b175a` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:41.114850 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b45f17-1413-4d94-94c8-bac0f61f5721 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65b94d2d675f4b30b907579b9a6b175a)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:41.116841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b45f17-1413-4d94-94c8-bac0f61f5721 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:41.160240 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:41.167379 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c4b45f17-1413-4d94-94c8-bac0f61f5721 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65b94d2d675f4b30b907579b9a6b175a'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:41.168063 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c4b45f17-1413-4d94-94c8-bac0f61f5721 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65b94d2d675f4b30b907579b9a6b175a'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:41.176102 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c4b45f17-1413-4d94-94c8-bac0f61f5721 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65b94d2d675f4b30b907579b9a6b175a'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:41.188802 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c4b45f17-1413-4d94-94c8-bac0f61f5721 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65b94d2d675f4b30b907579b9a6b175a'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:41.196358 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c4b45f17-1413-4d94-94c8-bac0f61f5721 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65b94d2d675f4b30b907579b9a6b175a'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:41.202424 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1018/2029] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:28:41 2022] DELETE /identity/v3/users/65b94d2d675f4b30b907579b9a6b175a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:41.207395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45c254-3c50-4820-ba85-eed6ab4c57fe None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:41.207894 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45c254-3c50-4820-ba85-eed6ab4c57fe None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:41.208009 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45c254-3c50-4820-ba85-eed6ab4c57fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:41.228213 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-db45c254-3c50-4820-ba85-eed6ab4c57fe None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:41.229427 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-db45c254-3c50-4820-ba85-eed6ab4c57fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:41.235994 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:41.291877 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1012/2030] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:41.303898 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b45f17-1413-4d94-94c8-bac0f61f5721 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:41.305978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-c4b45f17-1413-4d94-94c8-bac0f61f5721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:41.374051 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b45f17-1413-4d94-94c8-bac0f61f5721 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:41.379235 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ad898-f1d2-4df4-89ac-15947fa0d525 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:41.379843 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ad898-f1d2-4df4-89ac-15947fa0d525 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:41.380159 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ad898-f1d2-4df4-89ac-15947fa0d525 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:41.380424 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ad898-f1d2-4df4-89ac-15947fa0d525 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:41.394991 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ad898-f1d2-4df4-89ac-15947fa0d525 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:41.397176 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ad898-f1d2-4df4-89ac-15947fa0d525 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:41.411247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1019/2031] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:28:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:28:41.718704 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45c254-3c50-4820-ba85-eed6ab4c57fe None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:41.720218 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-db45c254-3c50-4820-ba85-eed6ab4c57fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:41.732590 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45c254-3c50-4820-ba85-eed6ab4c57fe None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:41.738174 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b13bb-1932-4fe8-9b1f-c36082312d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:41.739062 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b13bb-1932-4fe8-9b1f-c36082312d29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:41.739332 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b13bb-1932-4fe8-9b1f-c36082312d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:41.739724 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b13bb-1932-4fe8-9b1f-c36082312d29 admin admin] PATH_INFO: `/v3/projects/48e2b00d845442298d89458d4c29c0d2` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:41.740994 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b13bb-1932-4fe8-9b1f-c36082312d29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48e2b00d845442298d89458d4c29c0d2)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:41.742932 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b13bb-1932-4fe8-9b1f-c36082312d29 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:41.792415 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-356b13bb-1932-4fe8-9b1f-c36082312d29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48e2b00d845442298d89458d4c29c0d2', 'request_id': 'req-356b13bb-1932-4fe8-9b1f-c36082312d29'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:41.799418 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-356b13bb-1932-4fe8-9b1f-c36082312d29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48e2b00d845442298d89458d4c29c0d2', 'request_id': 'req-356b13bb-1932-4fe8-9b1f-c36082312d29'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:41.808835 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-356b13bb-1932-4fe8-9b1f-c36082312d29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48e2b00d845442298d89458d4c29c0d2', 'request_id': 'req-356b13bb-1932-4fe8-9b1f-c36082312d29'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:41.820834 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-356b13bb-1932-4fe8-9b1f-c36082312d29 admin admin] The token cache is being invalidate because project 48e2b00d845442298d89458d4c29c0d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:28:41.821121 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-356b13bb-1932-4fe8-9b1f-c36082312d29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:41.821790 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-356b13bb-1932-4fe8-9b1f-c36082312d29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:41.823368 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1013/2032] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:28:41 2022] DELETE /identity/v3/projects/48e2b00d845442298d89458d4c29c0d2 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:41.827503 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ad898-f1d2-4df4-89ac-15947fa0d525 service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:41.837778 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ad898-f1d2-4df4-89ac-15947fa0d525 service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:41.894929 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d2e39-87f3-4fa8-8628-f2db52b0356d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:41.895564 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d2e39-87f3-4fa8-8628-f2db52b0356d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:41.895840 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d2e39-87f3-4fa8-8628-f2db52b0356d admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:41.896234 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d2e39-87f3-4fa8-8628-f2db52b0356d admin admin] PATH_INFO: `/v3/users/9aae900208d04f798e87b5da0feab672` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:41.897226 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d2e39-87f3-4fa8-8628-f2db52b0356d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aae900208d04f798e87b5da0feab672)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:41.899148 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d2e39-87f3-4fa8-8628-f2db52b0356d admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:41.948992 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-084d2e39-87f3-4fa8-8628-f2db52b0356d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aae900208d04f798e87b5da0feab672'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:41.949706 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-084d2e39-87f3-4fa8-8628-f2db52b0356d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aae900208d04f798e87b5da0feab672'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:41.959729 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-084d2e39-87f3-4fa8-8628-f2db52b0356d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aae900208d04f798e87b5da0feab672'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:41.965132 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:41.978855 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-084d2e39-87f3-4fa8-8628-f2db52b0356d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aae900208d04f798e87b5da0feab672'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:41.986665 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-084d2e39-87f3-4fa8-8628-f2db52b0356d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aae900208d04f798e87b5da0feab672'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:41.992858 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1020/2033] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:28:41 2022] DELETE /identity/v3/users/9aae900208d04f798e87b5da0feab672 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:42.285021 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b13bb-1932-4fe8-9b1f-c36082312d29 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:42.287034 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-356b13bb-1932-4fe8-9b1f-c36082312d29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:42.308315 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b13bb-1932-4fe8-9b1f-c36082312d29 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:42.372825 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50358df-dac6-4719-b5a7-77a03670af9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:42.373474 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50358df-dac6-4719-b5a7-77a03670af9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:42.373750 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50358df-dac6-4719-b5a7-77a03670af9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:42.374019 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50358df-dac6-4719-b5a7-77a03670af9b admin admin] PATH_INFO: `/v3/projects/da1534be16584d9b88c5e3d62736b273` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:42.375236 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50358df-dac6-4719-b5a7-77a03670af9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=da1534be16584d9b88c5e3d62736b273)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:42.378485 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50358df-dac6-4719-b5a7-77a03670af9b admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:42.418519 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:42.430108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d50358df-dac6-4719-b5a7-77a03670af9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da1534be16584d9b88c5e3d62736b273', 'request_id': 'req-d50358df-dac6-4719-b5a7-77a03670af9b'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:42.436691 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d50358df-dac6-4719-b5a7-77a03670af9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da1534be16584d9b88c5e3d62736b273', 'request_id': 'req-d50358df-dac6-4719-b5a7-77a03670af9b'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:42.445480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d50358df-dac6-4719-b5a7-77a03670af9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da1534be16584d9b88c5e3d62736b273', 'request_id': 'req-d50358df-dac6-4719-b5a7-77a03670af9b'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:42.459477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d50358df-dac6-4719-b5a7-77a03670af9b admin admin] The token cache is being invalidate because project da1534be16584d9b88c5e3d62736b273 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:28:42.459781 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d50358df-dac6-4719-b5a7-77a03670af9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:42.460503 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d50358df-dac6-4719-b5a7-77a03670af9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:42.462331 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1014/2034] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:28:42 2022] DELETE /identity/v3/projects/da1534be16584d9b88c5e3d62736b273 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:42.467137 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7736e7-142b-4fd9-8ee7-80dd9790331c None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:42.467672 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7736e7-142b-4fd9-8ee7-80dd9790331c None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:42.467672 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7736e7-142b-4fd9-8ee7-80dd9790331c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:42.489187 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-cb7736e7-142b-4fd9-8ee7-80dd9790331c None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:42.490625 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-cb7736e7-142b-4fd9-8ee7-80dd9790331c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:42.567372 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1021/2035] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:42.575387 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce72a82a-88b3-4063-9767-30133dd6ac2e None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:42.575670 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce72a82a-88b3-4063-9767-30133dd6ac2e None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:42.576054 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce72a82a-88b3-4063-9767-30133dd6ac2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:42.596199 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-ce72a82a-88b3-4063-9767-30133dd6ac2e None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:42.597809 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-ce72a82a-88b3-4063-9767-30133dd6ac2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:42.606430 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1015/2036] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:42.610805 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7736e7-142b-4fd9-8ee7-80dd9790331c None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:42.620845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7736e7-142b-4fd9-8ee7-80dd9790331c None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:42.626007 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ec2a0-4554-4f06-ac6b-d82a683b08bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:42.626620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ec2a0-4554-4f06-ac6b-d82a683b08bb admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:42.626895 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ec2a0-4554-4f06-ac6b-d82a683b08bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:42.627163 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ec2a0-4554-4f06-ac6b-d82a683b08bb admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:42.627806 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ec2a0-4554-4f06-ac6b-d82a683b08bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:28:42.628139 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ec2a0-4554-4f06-ac6b-d82a683b08bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:42.629724 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ec2a0-4554-4f06-ac6b-d82a683b08bb admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:42.642101 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1022/2037] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:28:42.646587 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce72a82a-88b3-4063-9767-30133dd6ac2e None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:42.656340 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce72a82a-88b3-4063-9767-30133dd6ac2e None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:42.662059 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d45db0-78ba-47dd-a388-de206adf584d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:42.662792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d45db0-78ba-47dd-a388-de206adf584d admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:42.663401 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d45db0-78ba-47dd-a388-de206adf584d admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:42.663530 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d45db0-78ba-47dd-a388-de206adf584d admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:42.664301 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d45db0-78ba-47dd-a388-de206adf584d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:42.666349 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d45db0-78ba-47dd-a388-de206adf584d admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:42.707874 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1016/2038] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:28:42 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:28:42.709242 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ec2a0-4554-4f06-ac6b-d82a683b08bb admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:42.719857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ec2a0-4554-4f06-ac6b-d82a683b08bb admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:42.790184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779fe65e-488d-49f6-be7e-ee7b5577d823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:42.790964 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779fe65e-488d-49f6-be7e-ee7b5577d823 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:42.791319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779fe65e-488d-49f6-be7e-ee7b5577d823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:42.791657 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779fe65e-488d-49f6-be7e-ee7b5577d823 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:42.792412 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779fe65e-488d-49f6-be7e-ee7b5577d823 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:42.795110 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779fe65e-488d-49f6-be7e-ee7b5577d823 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:42.822602 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1023/2039] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:28:42 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:28:42.826521 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d45db0-78ba-47dd-a388-de206adf584d admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:42.837872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d45db0-78ba-47dd-a388-de206adf584d admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:42.842968 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f1f14b-546f-4c8b-bfaf-303a6f9aa7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:42.843581 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1f14b-546f-4c8b-bfaf-303a6f9aa7e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:42.843860 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1f14b-546f-4c8b-bfaf-303a6f9aa7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:42.844224 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1f14b-546f-4c8b-bfaf-303a6f9aa7e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:42.844911 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f1f14b-546f-4c8b-bfaf-303a6f9aa7e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:42.846474 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f1f14b-546f-4c8b-bfaf-303a6f9aa7e2 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:42.857365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1017/2040] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:42.861519 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779fe65e-488d-49f6-be7e-ee7b5577d823 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:42.871458 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779fe65e-488d-49f6-be7e-ee7b5577d823 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:42.876596 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f1533-10c7-4db4-800c-3d57d8c27a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:42.877205 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f1533-10c7-4db4-800c-3d57d8c27a84 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:42.877484 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f1533-10c7-4db4-800c-3d57d8c27a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:42.877751 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f1533-10c7-4db4-800c-3d57d8c27a84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:42.878425 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244f1533-10c7-4db4-800c-3d57d8c27a84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:42.880050 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244f1533-10c7-4db4-800c-3d57d8c27a84 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:42.892479 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1024/2041] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:42.896774 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f1f14b-546f-4c8b-bfaf-303a6f9aa7e2 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:42.909927 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f1f14b-546f-4c8b-bfaf-303a6f9aa7e2 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:42.916973 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd17828c-f183-4ee0-a802-dd51a3b186f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:42.917648 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17828c-f183-4ee0-a802-dd51a3b186f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:42.917924 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17828c-f183-4ee0-a802-dd51a3b186f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:42.918194 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17828c-f183-4ee0-a802-dd51a3b186f7 admin admin] PATH_INFO: `/v3/projects/aec8505d96be450188e176d091286fc4/users/9cd4aa1f6a4141289a6ae570fb4db6f1/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:42.935481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd17828c-f183-4ee0-a802-dd51a3b186f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aec8505d96be450188e176d091286fc4, user_id=9cd4aa1f6a4141289a6ae570fb4db6f1, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:42.938476 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd17828c-f183-4ee0-a802-dd51a3b186f7 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:42.956477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1018/2042] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:28:42 2022] PUT /identity/v3/projects/aec8505d96be450188e176d091286fc4/users/9cd4aa1f6a4141289a6ae570fb4db6f1/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:42.965701 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48b994-3699-4e1c-ac9a-62e70c898f80 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:42.966093 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48b994-3699-4e1c-ac9a-62e70c898f80 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:42.966499 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48b994-3699-4e1c-ac9a-62e70c898f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:43.006323 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-df48b994-3699-4e1c-ac9a-62e70c898f80 None None] MFA Rules not processed for user `9cd4aa1f6a4141289a6ae570fb4db6f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:43.007551 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-df48b994-3699-4e1c-ac9a-62e70c898f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:43.048784 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:43.091521 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1025/2043] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:43.105081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd17828c-f183-4ee0-a802-dd51a3b186f7 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:43.106802 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-bd17828c-f183-4ee0-a802-dd51a3b186f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:43.180838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd17828c-f183-4ee0-a802-dd51a3b186f7 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:43.185691 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b75d7-3fbb-4072-9849-76a9a10e191c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:43.187018 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b75d7-3fbb-4072-9849-76a9a10e191c service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:43.187751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b75d7-3fbb-4072-9849-76a9a10e191c service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:43.188260 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b75d7-3fbb-4072-9849-76a9a10e191c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:43.203278 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8b75d7-3fbb-4072-9849-76a9a10e191c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:43.208069 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8b75d7-3fbb-4072-9849-76a9a10e191c service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:43.215899 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1019/2044] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:28:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:28:44.718935 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24871d-792e-4e05-9f35-e90eef6013b2 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:44.719556 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24871d-792e-4e05-9f35-e90eef6013b2 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:44.719756 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24871d-792e-4e05-9f35-e90eef6013b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:44.740681 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-7b24871d-792e-4e05-9f35-e90eef6013b2 None None] MFA Rules not processed for user `9cd4aa1f6a4141289a6ae570fb4db6f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:44.741896 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-7b24871d-792e-4e05-9f35-e90eef6013b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:44.749720 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1026/2045] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:50.894342 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2979f-5438-4f40-9504-87ca89a51147 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:50.895035 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2979f-5438-4f40-9504-87ca89a51147 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:50.895035 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2979f-5438-4f40-9504-87ca89a51147 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:50.896488 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1020/2046] 10.222.0.18 () {54 vars in 921 bytes} [Tue Aug 23 04:28:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:28:50.906347 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74e97c-798b-4ade-b547-6681dda13596 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:50.907129 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74e97c-798b-4ade-b547-6681dda13596 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:50.907418 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74e97c-798b-4ade-b547-6681dda13596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:50.930457 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-6c74e97c-798b-4ade-b547-6681dda13596 None None] MFA Rules not processed for user `dd503fe3cf6d428e962c14ab864ee608`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:50.931837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-6c74e97c-798b-4ade-b547-6681dda13596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:51.001358 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2979f-5438-4f40-9504-87ca89a51147 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:51.001877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1027/2047] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Aug 23 04:28:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:51.003247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-00e2979f-5438-4f40-9504-87ca89a51147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:51.020596 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74e97c-798b-4ade-b547-6681dda13596 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:51.022022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-6c74e97c-798b-4ade-b547-6681dda13596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:51.037186 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74e97c-798b-4ade-b547-6681dda13596 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:51.042814 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3f71d-b6d4-4d78-a381-cf5532305885 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd503fe3cf6d428e962c14ab864ee608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:51.043639 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3f71d-b6d4-4d78-a381-cf5532305885 service placement] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:51.044010 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3f71d-b6d4-4d78-a381-cf5532305885 service placement] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:51.044354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3f71d-b6d4-4d78-a381-cf5532305885 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:51.054402 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d3f71d-b6d4-4d78-a381-cf5532305885 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:51.056027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d3f71d-b6d4-4d78-a381-cf5532305885 service placement] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:51.066854 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1028/2048] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:28:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:28:51.090384 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2979f-5438-4f40-9504-87ca89a51147 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:51.095923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32280de-6f9b-44b8-b92c-bf07c9ab89d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:51.096794 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32280de-6f9b-44b8-b92c-bf07c9ab89d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:51.097073 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32280de-6f9b-44b8-b92c-bf07c9ab89d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:51.097666 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32280de-6f9b-44b8-b92c-bf07c9ab89d0 admin admin] PATH_INFO: `/v3/users/bafdb895190e45ba96e71fd348c41c3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:51.099063 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32280de-6f9b-44b8-b92c-bf07c9ab89d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bafdb895190e45ba96e71fd348c41c3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:51.100937 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32280de-6f9b-44b8-b92c-bf07c9ab89d0 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:51.104030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:51.163237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-c32280de-6f9b-44b8-b92c-bf07c9ab89d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bafdb895190e45ba96e71fd348c41c3f'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:51.173636 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-c32280de-6f9b-44b8-b92c-bf07c9ab89d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bafdb895190e45ba96e71fd348c41c3f'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:51.186854 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-c32280de-6f9b-44b8-b92c-bf07c9ab89d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bafdb895190e45ba96e71fd348c41c3f'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:51.187572 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-c32280de-6f9b-44b8-b92c-bf07c9ab89d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bafdb895190e45ba96e71fd348c41c3f'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:51.194865 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-c32280de-6f9b-44b8-b92c-bf07c9ab89d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bafdb895190e45ba96e71fd348c41c3f'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:51.201088 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1021/2049] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:28:51 2022] DELETE /identity/v3/users/bafdb895190e45ba96e71fd348c41c3f => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:51.387632 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3f71d-b6d4-4d78-a381-cf5532305885 service placement] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:51.400503 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3f71d-b6d4-4d78-a381-cf5532305885 service placement] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:51.461513 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c75202-7f4d-4730-92a3-4b4a388687a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:51.462179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c75202-7f4d-4730-92a3-4b4a388687a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:51.462479 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c75202-7f4d-4730-92a3-4b4a388687a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:51.462757 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c75202-7f4d-4730-92a3-4b4a388687a3 admin admin] PATH_INFO: `/v3/projects/3d3169d3b318421fab1d473a51637437` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:51.463849 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c75202-7f4d-4730-92a3-4b4a388687a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d3169d3b318421fab1d473a51637437)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:51.465796 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c75202-7f4d-4730-92a3-4b4a388687a3 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:51.513916 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c5c75202-7f4d-4730-92a3-4b4a388687a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d3169d3b318421fab1d473a51637437', 'request_id': 'req-c5c75202-7f4d-4730-92a3-4b4a388687a3'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:51.515709 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:51.529068 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c5c75202-7f4d-4730-92a3-4b4a388687a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d3169d3b318421fab1d473a51637437', 'request_id': 'req-c5c75202-7f4d-4730-92a3-4b4a388687a3'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:51.539799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c5c75202-7f4d-4730-92a3-4b4a388687a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d3169d3b318421fab1d473a51637437', 'request_id': 'req-c5c75202-7f4d-4730-92a3-4b4a388687a3'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:51.551692 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c5c75202-7f4d-4730-92a3-4b4a388687a3 admin admin] The token cache is being invalidate because project 3d3169d3b318421fab1d473a51637437 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:28:51.552001 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c5c75202-7f4d-4730-92a3-4b4a388687a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:51.552645 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c5c75202-7f4d-4730-92a3-4b4a388687a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:51.554153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1029/2050] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:28:51 2022] DELETE /identity/v3/projects/3d3169d3b318421fab1d473a51637437 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:51.559165 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e0c73-1cb9-4222-9b0e-acef0d1ff7cc None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:51.559459 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e0c73-1cb9-4222-9b0e-acef0d1ff7cc None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:51.559740 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e0c73-1cb9-4222-9b0e-acef0d1ff7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:51.580646 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-a59e0c73-1cb9-4222-9b0e-acef0d1ff7cc None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:51.581795 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-a59e0c73-1cb9-4222-9b0e-acef0d1ff7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:51.646655 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1022/2051] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:51.654515 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6145b-5ffc-4b30-981c-631c78c0b326 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:51.654796 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6145b-5ffc-4b30-981c-631c78c0b326 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:51.655068 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6145b-5ffc-4b30-981c-631c78c0b326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:51.676179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-efb6145b-5ffc-4b30-981c-631c78c0b326 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:51.677004 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-efb6145b-5ffc-4b30-981c-631c78c0b326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:51.687191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1030/2052] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:51.691847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e0c73-1cb9-4222-9b0e-acef0d1ff7cc None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:51.702054 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e0c73-1cb9-4222-9b0e-acef0d1ff7cc None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:51.706766 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb244a-36bc-4dde-bab6-12fd80693717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:51.707384 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb244a-36bc-4dde-bab6-12fd80693717 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:51.707659 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb244a-36bc-4dde-bab6-12fd80693717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:51.707939 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb244a-36bc-4dde-bab6-12fd80693717 admin admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:51.708586 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb244a-36bc-4dde-bab6-12fd80693717 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:28:51.708889 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb244a-36bc-4dde-bab6-12fd80693717 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:51.710557 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb244a-36bc-4dde-bab6-12fd80693717 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:51.722968 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1023/2053] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:28:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:28:51.728969 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6145b-5ffc-4b30-981c-631c78c0b326 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:51.739127 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6145b-5ffc-4b30-981c-631c78c0b326 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:51.744886 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea03a74-97ef-4d5c-a223-bc105716aa9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:51.745499 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea03a74-97ef-4d5c-a223-bc105716aa9e admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:51.745775 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea03a74-97ef-4d5c-a223-bc105716aa9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:51.746043 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea03a74-97ef-4d5c-a223-bc105716aa9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:51.746668 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea03a74-97ef-4d5c-a223-bc105716aa9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:51.748664 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea03a74-97ef-4d5c-a223-bc105716aa9e admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:51.778558 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1031/2054] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:28:51 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:28:51.782892 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb244a-36bc-4dde-bab6-12fd80693717 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:51.793383 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb244a-36bc-4dde-bab6-12fd80693717 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:51.857953 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c6504-4d38-4039-bb1a-1191f87d2ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:51.858476 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c6504-4d38-4039-bb1a-1191f87d2ca0 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:51.858757 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c6504-4d38-4039-bb1a-1191f87d2ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:51.859054 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c6504-4d38-4039-bb1a-1191f87d2ca0 admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:51.859824 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c6504-4d38-4039-bb1a-1191f87d2ca0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:51.861881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c6504-4d38-4039-bb1a-1191f87d2ca0 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:51.879940 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:51.886505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1024/2055] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:28:51 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:28:51.890531 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea03a74-97ef-4d5c-a223-bc105716aa9e admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:51.901641 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea03a74-97ef-4d5c-a223-bc105716aa9e admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:51.906902 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d6ba1-9c00-4b8e-aed6-72ccdffd73f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:51.907764 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d6ba1-9c00-4b8e-aed6-72ccdffd73f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:51.908081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d6ba1-9c00-4b8e-aed6-72ccdffd73f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:51.908328 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d6ba1-9c00-4b8e-aed6-72ccdffd73f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:51.909083 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2d6ba1-9c00-4b8e-aed6-72ccdffd73f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:51.910972 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2d6ba1-9c00-4b8e-aed6-72ccdffd73f4 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:51.921880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1032/2056] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:28:51.926002 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c6504-4d38-4039-bb1a-1191f87d2ca0 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:51.936660 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c6504-4d38-4039-bb1a-1191f87d2ca0 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:51.942117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5c109-d050-417b-9f05-f0b88ccbb6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:51.942729 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5c109-d050-417b-9f05-f0b88ccbb6eb admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:51.942999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5c109-d050-417b-9f05-f0b88ccbb6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:51.943262 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5c109-d050-417b-9f05-f0b88ccbb6eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:51.943906 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5c109-d050-417b-9f05-f0b88ccbb6eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:51.945555 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5c109-d050-417b-9f05-f0b88ccbb6eb admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:51.963316 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1025/2057] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:28:51.964628 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d6ba1-9c00-4b8e-aed6-72ccdffd73f4 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:51.975294 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d6ba1-9c00-4b8e-aed6-72ccdffd73f4 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:51.979943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734f185-42f5-45b6-af6e-90a42e11d139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:51.980608 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734f185-42f5-45b6-af6e-90a42e11d139 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:51.980885 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734f185-42f5-45b6-af6e-90a42e11d139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:51.981158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734f185-42f5-45b6-af6e-90a42e11d139 admin admin] PATH_INFO: `/v3/projects/003eb44fe9d44984a8f7bfaa093afab0/users/2179a4b92c9b47a08491e0b678accda4/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:51.997108 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0734f185-42f5-45b6-af6e-90a42e11d139 admin admin] RBAC: Authorizing `identity:create_grant(project_id=003eb44fe9d44984a8f7bfaa093afab0, user_id=2179a4b92c9b47a08491e0b678accda4, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:51.999938 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0734f185-42f5-45b6-af6e-90a42e11d139 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:52.013999 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1033/2058] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:28:51 2022] PUT /identity/v3/projects/003eb44fe9d44984a8f7bfaa093afab0/users/2179a4b92c9b47a08491e0b678accda4/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:52.024449 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8dde5-61d9-436b-b567-d8a1f285499e None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:52.024735 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8dde5-61d9-436b-b567-d8a1f285499e None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:52.024972 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8dde5-61d9-436b-b567-d8a1f285499e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:52.054397 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-b1e8dde5-61d9-436b-b567-d8a1f285499e None None] MFA Rules not processed for user `2179a4b92c9b47a08491e0b678accda4`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:52.055602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-b1e8dde5-61d9-436b-b567-d8a1f285499e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:52.124751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1026/2059] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:28:52.136108 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734f185-42f5-45b6-af6e-90a42e11d139 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:52.138292 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-0734f185-42f5-45b6-af6e-90a42e11d139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:52.213275 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734f185-42f5-45b6-af6e-90a42e11d139 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:52.219765 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c384d-3867-42c7-bf73-ed589ef3dcb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:52.220402 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c384d-3867-42c7-bf73-ed589ef3dcb2 service nova] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:52.220683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c384d-3867-42c7-bf73-ed589ef3dcb2 service nova] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:52.220983 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c384d-3867-42c7-bf73-ed589ef3dcb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:52.229690 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c384d-3867-42c7-bf73-ed589ef3dcb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:52.231281 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c384d-3867-42c7-bf73-ed589ef3dcb2 service nova] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:52.240537 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1034/2060] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:28:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:28:54.946552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e8dde5-61d9-436b-b567-d8a1f285499e None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:55.050000 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e8dde5-61d9-436b-b567-d8a1f285499e None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:55.054485 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab0aac-faff-41cb-8bd3-df685f6edfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:55.055123 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab0aac-faff-41cb-8bd3-df685f6edfb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:55.055402 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab0aac-faff-41cb-8bd3-df685f6edfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:55.055676 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab0aac-faff-41cb-8bd3-df685f6edfb8 admin admin] PATH_INFO: `/v3/users/2179a4b92c9b47a08491e0b678accda4` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:55.056811 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab0aac-faff-41cb-8bd3-df685f6edfb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2179a4b92c9b47a08491e0b678accda4)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:55.058758 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab0aac-faff-41cb-8bd3-df685f6edfb8 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:55.108445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9fab0aac-faff-41cb-8bd3-df685f6edfb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2179a4b92c9b47a08491e0b678accda4'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:55.118183 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9fab0aac-faff-41cb-8bd3-df685f6edfb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2179a4b92c9b47a08491e0b678accda4'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:55.134293 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9fab0aac-faff-41cb-8bd3-df685f6edfb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2179a4b92c9b47a08491e0b678accda4'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:55.135292 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9fab0aac-faff-41cb-8bd3-df685f6edfb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2179a4b92c9b47a08491e0b678accda4'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:55.141140 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:55.143429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9fab0aac-faff-41cb-8bd3-df685f6edfb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2179a4b92c9b47a08491e0b678accda4'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:55.152380 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1027/2061] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:28:54 2022] DELETE /identity/v3/users/2179a4b92c9b47a08491e0b678accda4 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:55.158164 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578bb81-0fdb-4240-8d3d-3b0427102ec5 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:55.158689 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578bb81-0fdb-4240-8d3d-3b0427102ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:55.158689 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578bb81-0fdb-4240-8d3d-3b0427102ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:55.180003 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-7578bb81-0fdb-4240-8d3d-3b0427102ec5 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:55.181335 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-7578bb81-0fdb-4240-8d3d-3b0427102ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:55.248835 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1035/2062] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:28:55.258872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab0aac-faff-41cb-8bd3-df685f6edfb8 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:55.260522 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-9fab0aac-faff-41cb-8bd3-df685f6edfb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:55.341353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab0aac-faff-41cb-8bd3-df685f6edfb8 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:55.345801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5236d-853f-4d41-8768-857a16ba2f77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:55.346430 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5236d-853f-4d41-8768-857a16ba2f77 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:55.346711 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5236d-853f-4d41-8768-857a16ba2f77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:55.346985 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5236d-853f-4d41-8768-857a16ba2f77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:55.357121 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb5236d-853f-4d41-8768-857a16ba2f77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:55.358755 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb5236d-853f-4d41-8768-857a16ba2f77 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:55.368400 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1028/2063] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:28:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:28:55.663928 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578bb81-0fdb-4240-8d3d-3b0427102ec5 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:55.666344 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-7578bb81-0fdb-4240-8d3d-3b0427102ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:55.678054 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:55.686918 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578bb81-0fdb-4240-8d3d-3b0427102ec5 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:55.696112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999dfde7-2214-4ae9-ab0e-85bcc797a51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:55.697323 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999dfde7-2214-4ae9-ab0e-85bcc797a51b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:55.697949 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999dfde7-2214-4ae9-ab0e-85bcc797a51b admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:55.698444 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999dfde7-2214-4ae9-ab0e-85bcc797a51b admin admin] PATH_INFO: `/v3/projects/003eb44fe9d44984a8f7bfaa093afab0` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:55.700123 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999dfde7-2214-4ae9-ab0e-85bcc797a51b admin admin] RBAC: Authorizing `identity:delete_project(project_id=003eb44fe9d44984a8f7bfaa093afab0)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:55.702092 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999dfde7-2214-4ae9-ab0e-85bcc797a51b admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:55.768966 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-999dfde7-2214-4ae9-ab0e-85bcc797a51b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '003eb44fe9d44984a8f7bfaa093afab0', 'request_id': 'req-999dfde7-2214-4ae9-ab0e-85bcc797a51b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:55.776111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-999dfde7-2214-4ae9-ab0e-85bcc797a51b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '003eb44fe9d44984a8f7bfaa093afab0', 'request_id': 'req-999dfde7-2214-4ae9-ab0e-85bcc797a51b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:55.785574 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-999dfde7-2214-4ae9-ab0e-85bcc797a51b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '003eb44fe9d44984a8f7bfaa093afab0', 'request_id': 'req-999dfde7-2214-4ae9-ab0e-85bcc797a51b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:55.796669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-999dfde7-2214-4ae9-ab0e-85bcc797a51b admin admin] The token cache is being invalidate because project 003eb44fe9d44984a8f7bfaa093afab0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:28:55.796954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-999dfde7-2214-4ae9-ab0e-85bcc797a51b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:55.798253 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-999dfde7-2214-4ae9-ab0e-85bcc797a51b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:55.799815 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1036/2064] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:28:55 2022] DELETE /identity/v3/projects/003eb44fe9d44984a8f7bfaa093afab0 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:55.804803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fce1e-b75e-49f8-a341-770ee0de630e None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:55.805120 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fce1e-b75e-49f8-a341-770ee0de630e None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:55.805404 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fce1e-b75e-49f8-a341-770ee0de630e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:55.826751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-a08fce1e-b75e-49f8-a341-770ee0de630e None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:55.827990 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-a08fce1e-b75e-49f8-a341-770ee0de630e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:55.897712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1029/2065] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:55.905725 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5db953-c72b-4a5b-9492-40f9d95df270 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:55.906008 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5db953-c72b-4a5b-9492-40f9d95df270 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:55.906283 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5db953-c72b-4a5b-9492-40f9d95df270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:55.926585 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-bc5db953-c72b-4a5b-9492-40f9d95df270 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:55.927791 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-bc5db953-c72b-4a5b-9492-40f9d95df270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:55.936518 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1037/2066] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:55.940920 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fce1e-b75e-49f8-a341-770ee0de630e None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:55.951567 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fce1e-b75e-49f8-a341-770ee0de630e None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:55.957187 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7990bd-adcd-4282-b9de-75ea0ea91501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:55.957997 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7990bd-adcd-4282-b9de-75ea0ea91501 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:55.958369 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7990bd-adcd-4282-b9de-75ea0ea91501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:55.958663 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7990bd-adcd-4282-b9de-75ea0ea91501 admin admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:55.959425 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7990bd-adcd-4282-b9de-75ea0ea91501 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:28:55.959856 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7990bd-adcd-4282-b9de-75ea0ea91501 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:55.961871 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7990bd-adcd-4282-b9de-75ea0ea91501 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:55.974934 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1030/2067] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:28:55.979654 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5db953-c72b-4a5b-9492-40f9d95df270 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:55.989703 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5db953-c72b-4a5b-9492-40f9d95df270 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:55.995764 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd85443-2413-47bd-827f-16e58180acdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:55.996382 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd85443-2413-47bd-827f-16e58180acdd admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:55.996657 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd85443-2413-47bd-827f-16e58180acdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:55.996920 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd85443-2413-47bd-827f-16e58180acdd admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:55.997537 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd85443-2413-47bd-827f-16e58180acdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:55.999439 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd85443-2413-47bd-827f-16e58180acdd admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:56.036485 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1038/2068] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:28:55 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:28:56.038996 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7990bd-adcd-4282-b9de-75ea0ea91501 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:56.055114 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7990bd-adcd-4282-b9de-75ea0ea91501 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:56.122960 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86d908-69a4-420e-8185-9910cba12055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:56.123670 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86d908-69a4-420e-8185-9910cba12055 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:56.123973 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86d908-69a4-420e-8185-9910cba12055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:56.124246 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86d908-69a4-420e-8185-9910cba12055 admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:56.125124 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b86d908-69a4-420e-8185-9910cba12055 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:56.127785 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b86d908-69a4-420e-8185-9910cba12055 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:56.152949 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1031/2069] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:28:56 2022] POST /identity/v3/users => generated 294 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:28:56.157060 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd85443-2413-47bd-827f-16e58180acdd admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:56.167523 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd85443-2413-47bd-827f-16e58180acdd admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:56.172362 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b4702-0e97-4ea7-897a-1bdee7ba8977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:56.172975 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b4702-0e97-4ea7-897a-1bdee7ba8977 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:56.173252 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b4702-0e97-4ea7-897a-1bdee7ba8977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:56.173519 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b4702-0e97-4ea7-897a-1bdee7ba8977 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:56.174135 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334b4702-0e97-4ea7-897a-1bdee7ba8977 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:56.175723 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334b4702-0e97-4ea7-897a-1bdee7ba8977 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:56.185821 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1039/2070] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:56.190490 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86d908-69a4-420e-8185-9910cba12055 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:56.201111 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86d908-69a4-420e-8185-9910cba12055 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:56.207070 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea45986-29f5-49b4-9f28-b885e5120a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:56.207676 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea45986-29f5-49b4-9f28-b885e5120a78 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:56.207928 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea45986-29f5-49b4-9f28-b885e5120a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:56.208245 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea45986-29f5-49b4-9f28-b885e5120a78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:56.208880 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea45986-29f5-49b4-9f28-b885e5120a78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:56.210479 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea45986-29f5-49b4-9f28-b885e5120a78 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:56.221966 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1032/2071] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:28:56.226476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b4702-0e97-4ea7-897a-1bdee7ba8977 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:56.237088 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b4702-0e97-4ea7-897a-1bdee7ba8977 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:56.242014 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06d2d9-9d86-49d7-9849-418a3e9a8e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:56.242647 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06d2d9-9d86-49d7-9849-418a3e9a8e65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:56.242916 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06d2d9-9d86-49d7-9849-418a3e9a8e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:56.243182 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06d2d9-9d86-49d7-9849-418a3e9a8e65 admin admin] PATH_INFO: `/v3/projects/0f465c4269b843fe87a00685ffc9794d/users/ec45e054e27b4103af0ea017a2e821ad/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:56.258631 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe06d2d9-9d86-49d7-9849-418a3e9a8e65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f465c4269b843fe87a00685ffc9794d, user_id=ec45e054e27b4103af0ea017a2e821ad, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:56.261281 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe06d2d9-9d86-49d7-9849-418a3e9a8e65 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:56.278395 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1040/2072] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:28:56 2022] PUT /identity/v3/projects/0f465c4269b843fe87a00685ffc9794d/users/ec45e054e27b4103af0ea017a2e821ad/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:56.291836 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a55f3-c334-4b78-9495-bcf39c9956ea None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:56.292346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a55f3-c334-4b78-9495-bcf39c9956ea None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:56.292912 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a55f3-c334-4b78-9495-bcf39c9956ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:56.303372 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:56.324327 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-194a55f3-c334-4b78-9495-bcf39c9956ea None None] MFA Rules not processed for user `ec45e054e27b4103af0ea017a2e821ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:56.325547 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-194a55f3-c334-4b78-9495-bcf39c9956ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:56.402312 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1033/2073] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:56.406982 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06d2d9-9d86-49d7-9849-418a3e9a8e65 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:56.478539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06d2d9-9d86-49d7-9849-418a3e9a8e65 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:56.483422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fc1c0-f310-4231-b111-60da1f30c088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:56.484191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fc1c0-f310-4231-b111-60da1f30c088 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:56.484560 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fc1c0-f310-4231-b111-60da1f30c088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:56.485006 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fc1c0-f310-4231-b111-60da1f30c088 admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:56.485837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43fc1c0-f310-4231-b111-60da1f30c088 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:56.487851 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43fc1c0-f310-4231-b111-60da1f30c088 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:56.519854 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1041/2074] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:28:56 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:28:56.521195 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a55f3-c334-4b78-9495-bcf39c9956ea None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:56.533637 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a55f3-c334-4b78-9495-bcf39c9956ea None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:56.600298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c407fd-c7ca-4b02-9d72-2cc819715d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:56.600924 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c407fd-c7ca-4b02-9d72-2cc819715d86 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:56.601179 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c407fd-c7ca-4b02-9d72-2cc819715d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:56.601473 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c407fd-c7ca-4b02-9d72-2cc819715d86 admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:56.602103 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c407fd-c7ca-4b02-9d72-2cc819715d86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:56.604065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c407fd-c7ca-4b02-9d72-2cc819715d86 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:56.634250 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1034/2075] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:28:56 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:28:56.635762 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fc1c0-f310-4231-b111-60da1f30c088 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:56.648074 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fc1c0-f310-4231-b111-60da1f30c088 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:56.653111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c9673-72af-4ea4-9ffd-a6c165797670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:56.653812 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c9673-72af-4ea4-9ffd-a6c165797670 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:56.654160 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c9673-72af-4ea4-9ffd-a6c165797670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:56.654517 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c9673-72af-4ea4-9ffd-a6c165797670 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:56.655236 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c9673-72af-4ea4-9ffd-a6c165797670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:56.657118 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c9673-72af-4ea4-9ffd-a6c165797670 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:56.661114 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:56.675429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c407fd-c7ca-4b02-9d72-2cc819715d86 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:56.676124 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1042/2076] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:56.686013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c407fd-c7ca-4b02-9d72-2cc819715d86 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:56.690829 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47a1c7-61f5-4613-818d-50a499ea0de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:56.691675 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47a1c7-61f5-4613-818d-50a499ea0de8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:56.692052 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47a1c7-61f5-4613-818d-50a499ea0de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:56.692416 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47a1c7-61f5-4613-818d-50a499ea0de8 admin admin] PATH_INFO: `/v3/projects/9f084d2dae774495a8eb5ff10ab7d2f9/users/ca09585dc6ad4dd4940db0b0480a318f/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:56.715112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a47a1c7-61f5-4613-818d-50a499ea0de8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f084d2dae774495a8eb5ff10ab7d2f9, user_id=ca09585dc6ad4dd4940db0b0480a318f, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:56.718598 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a47a1c7-61f5-4613-818d-50a499ea0de8 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:56.737744 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1035/2077] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:28:56 2022] PUT /identity/v3/projects/9f084d2dae774495a8eb5ff10ab7d2f9/users/ca09585dc6ad4dd4940db0b0480a318f/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:56.741933 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c9673-72af-4ea4-9ffd-a6c165797670 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:56.752966 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c9673-72af-4ea4-9ffd-a6c165797670 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:56.831192 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99908107-b99e-4b4f-aba5-20e043ab51ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:56.831784 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99908107-b99e-4b4f-aba5-20e043ab51ce admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:56.832141 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99908107-b99e-4b4f-aba5-20e043ab51ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:56.832439 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99908107-b99e-4b4f-aba5-20e043ab51ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:56.833095 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99908107-b99e-4b4f-aba5-20e043ab51ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:56.834810 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99908107-b99e-4b4f-aba5-20e043ab51ce admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:56.850928 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1043/2078] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:56.852215 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47a1c7-61f5-4613-818d-50a499ea0de8 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:56.864969 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47a1c7-61f5-4613-818d-50a499ea0de8 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:56.869618 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cbd9b6-0bb6-4986-ad38-ea16c5cd8001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:56.870275 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cbd9b6-0bb6-4986-ad38-ea16c5cd8001 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:56.870550 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cbd9b6-0bb6-4986-ad38-ea16c5cd8001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:56.870816 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cbd9b6-0bb6-4986-ad38-ea16c5cd8001 admin admin] PATH_INFO: `/v3/projects/9f084d2dae774495a8eb5ff10ab7d2f9/users/ca09585dc6ad4dd4940db0b0480a318f/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:56.872736 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cbd9b6-0bb6-4986-ad38-ea16c5cd8001 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f084d2dae774495a8eb5ff10ab7d2f9, user_id=ca09585dc6ad4dd4940db0b0480a318f, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:56.876322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cbd9b6-0bb6-4986-ad38-ea16c5cd8001 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:56.887276 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1036/2079] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:28:56 2022] PUT /identity/v3/projects/9f084d2dae774495a8eb5ff10ab7d2f9/users/ca09585dc6ad4dd4940db0b0480a318f/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:56.897872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59df56a-f80d-4cb7-81c6-93e2c9caba5e None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:56.898163 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59df56a-f80d-4cb7-81c6-93e2c9caba5e None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:56.898373 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59df56a-f80d-4cb7-81c6-93e2c9caba5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:56.929440 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-d59df56a-f80d-4cb7-81c6-93e2c9caba5e None None] MFA Rules not processed for user `ca09585dc6ad4dd4940db0b0480a318f`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:56.930654 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-d59df56a-f80d-4cb7-81c6-93e2c9caba5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:57.004023 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1044/2080] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:56 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:57.015979 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cbd9b6-0bb6-4986-ad38-ea16c5cd8001 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:57.018150 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-a7cbd9b6-0bb6-4986-ad38-ea16c5cd8001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:57.072958 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:57.081322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cbd9b6-0bb6-4986-ad38-ea16c5cd8001 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:57.087036 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc82f3-254f-4352-b5fb-57b810eedde4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:57.087991 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc82f3-254f-4352-b5fb-57b810eedde4 service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:57.088343 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc82f3-254f-4352-b5fb-57b810eedde4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:57.088990 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc82f3-254f-4352-b5fb-57b810eedde4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:57.106184 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc82f3-254f-4352-b5fb-57b810eedde4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:57.108453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc82f3-254f-4352-b5fb-57b810eedde4 service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:57.119214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1037/2081] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:28:57 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:28:57.184940 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59df56a-f80d-4cb7-81c6-93e2c9caba5e None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:57.255603 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59df56a-f80d-4cb7-81c6-93e2c9caba5e None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:57.260445 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1339f9-083e-4328-8cb4-b72fde276577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:57.261133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1339f9-083e-4328-8cb4-b72fde276577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:57.261427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1339f9-083e-4328-8cb4-b72fde276577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:57.261701 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1339f9-083e-4328-8cb4-b72fde276577 admin admin] PATH_INFO: `/v3/users/ec45e054e27b4103af0ea017a2e821ad` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:57.262881 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1339f9-083e-4328-8cb4-b72fde276577 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec45e054e27b4103af0ea017a2e821ad)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:57.264833 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1339f9-083e-4328-8cb4-b72fde276577 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:57.286803 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:57.315681 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-5b1339f9-083e-4328-8cb4-b72fde276577 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec45e054e27b4103af0ea017a2e821ad'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:57.316355 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-5b1339f9-083e-4328-8cb4-b72fde276577 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec45e054e27b4103af0ea017a2e821ad'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:57.325253 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-5b1339f9-083e-4328-8cb4-b72fde276577 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec45e054e27b4103af0ea017a2e821ad'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:57.337554 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-5b1339f9-083e-4328-8cb4-b72fde276577 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec45e054e27b4103af0ea017a2e821ad'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:57.344824 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-5b1339f9-083e-4328-8cb4-b72fde276577 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec45e054e27b4103af0ea017a2e821ad'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:57.351212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1045/2082] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:28:57 2022] DELETE /identity/v3/users/ec45e054e27b4103af0ea017a2e821ad => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:57.356071 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c1d2f-6220-4bd1-b13f-5a72f4bf12e3 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:57.356359 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c1d2f-6220-4bd1-b13f-5a72f4bf12e3 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:57.356623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c1d2f-6220-4bd1-b13f-5a72f4bf12e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:57.376785 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-f61c1d2f-6220-4bd1-b13f-5a72f4bf12e3 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:57.377997 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-f61c1d2f-6220-4bd1-b13f-5a72f4bf12e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:57.441241 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1038/2083] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:57.451116 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1339f9-083e-4328-8cb4-b72fde276577 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:57.452650 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-5b1339f9-083e-4328-8cb4-b72fde276577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:57.511201 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1339f9-083e-4328-8cb4-b72fde276577 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:57.515624 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052dfe2-a31c-4998-a24d-7a66bfb78a61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:57.516510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052dfe2-a31c-4998-a24d-7a66bfb78a61 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:57.516851 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052dfe2-a31c-4998-a24d-7a66bfb78a61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:57.517125 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052dfe2-a31c-4998-a24d-7a66bfb78a61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:57.528729 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052dfe2-a31c-4998-a24d-7a66bfb78a61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:57.530188 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052dfe2-a31c-4998-a24d-7a66bfb78a61 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:57.540715 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1046/2084] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:28:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:28:57.830120 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c1d2f-6220-4bd1-b13f-5a72f4bf12e3 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:57.831877 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-f61c1d2f-6220-4bd1-b13f-5a72f4bf12e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:57.851167 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c1d2f-6220-4bd1-b13f-5a72f4bf12e3 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:57.861703 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e75b2-3138-4cf1-953b-db0e6d1fb3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:57.862367 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e75b2-3138-4cf1-953b-db0e6d1fb3b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:57.862624 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e75b2-3138-4cf1-953b-db0e6d1fb3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:57.862944 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e75b2-3138-4cf1-953b-db0e6d1fb3b2 admin admin] PATH_INFO: `/v3/projects/0f465c4269b843fe87a00685ffc9794d` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:57.864261 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e75b2-3138-4cf1-953b-db0e6d1fb3b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f465c4269b843fe87a00685ffc9794d)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:57.866315 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e75b2-3138-4cf1-953b-db0e6d1fb3b2 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:57.925885 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-e80e75b2-3138-4cf1-953b-db0e6d1fb3b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f465c4269b843fe87a00685ffc9794d', 'request_id': 'req-e80e75b2-3138-4cf1-953b-db0e6d1fb3b2'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:57.932500 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-e80e75b2-3138-4cf1-953b-db0e6d1fb3b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f465c4269b843fe87a00685ffc9794d', 'request_id': 'req-e80e75b2-3138-4cf1-953b-db0e6d1fb3b2'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:57.941150 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-e80e75b2-3138-4cf1-953b-db0e6d1fb3b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f465c4269b843fe87a00685ffc9794d', 'request_id': 'req-e80e75b2-3138-4cf1-953b-db0e6d1fb3b2'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:57.953431 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-e80e75b2-3138-4cf1-953b-db0e6d1fb3b2 admin admin] The token cache is being invalidate because project 0f465c4269b843fe87a00685ffc9794d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:28:57.953893 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-e80e75b2-3138-4cf1-953b-db0e6d1fb3b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:57.955137 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-e80e75b2-3138-4cf1-953b-db0e6d1fb3b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:57.956800 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1039/2085] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:28:57 2022] DELETE /identity/v3/projects/0f465c4269b843fe87a00685ffc9794d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:57.961573 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052dfe2-a31c-4998-a24d-7a66bfb78a61 service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:57.971880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052dfe2-a31c-4998-a24d-7a66bfb78a61 service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:58.039369 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb425dd9-39ec-4c22-996d-3831650f341e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:58.040093 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb425dd9-39ec-4c22-996d-3831650f341e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:58.040465 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb425dd9-39ec-4c22-996d-3831650f341e admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:58.040745 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb425dd9-39ec-4c22-996d-3831650f341e admin admin] PATH_INFO: `/v3/users/ca09585dc6ad4dd4940db0b0480a318f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:58.041871 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb425dd9-39ec-4c22-996d-3831650f341e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca09585dc6ad4dd4940db0b0480a318f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:58.044062 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb425dd9-39ec-4c22-996d-3831650f341e admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:58.100416 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:58.108186 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-cb425dd9-39ec-4c22-996d-3831650f341e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca09585dc6ad4dd4940db0b0480a318f'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:58.110507 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-cb425dd9-39ec-4c22-996d-3831650f341e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca09585dc6ad4dd4940db0b0480a318f'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:58.118853 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-cb425dd9-39ec-4c22-996d-3831650f341e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca09585dc6ad4dd4940db0b0480a318f'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:58.132034 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-cb425dd9-39ec-4c22-996d-3831650f341e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca09585dc6ad4dd4940db0b0480a318f'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:58.138951 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-cb425dd9-39ec-4c22-996d-3831650f341e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca09585dc6ad4dd4940db0b0480a318f'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:58.145219 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1047/2086] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:28:57 2022] DELETE /identity/v3/users/ca09585dc6ad4dd4940db0b0480a318f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:58.441343 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e75b2-3138-4cf1-953b-db0e6d1fb3b2 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:58.443040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-e80e75b2-3138-4cf1-953b-db0e6d1fb3b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:58.470792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e75b2-3138-4cf1-953b-db0e6d1fb3b2 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:58.538384 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2e6ab1-ffaa-4988-9e62-bf39ec0e5a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:58.539024 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e6ab1-ffaa-4988-9e62-bf39ec0e5a2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:58.539299 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e6ab1-ffaa-4988-9e62-bf39ec0e5a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:58.539566 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e6ab1-ffaa-4988-9e62-bf39ec0e5a2c admin admin] PATH_INFO: `/v3/projects/9f084d2dae774495a8eb5ff10ab7d2f9` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:58.540801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2e6ab1-ffaa-4988-9e62-bf39ec0e5a2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f084d2dae774495a8eb5ff10ab7d2f9)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:58.542684 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2e6ab1-ffaa-4988-9e62-bf39ec0e5a2c admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:58.561255 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:58.594617 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-7f2e6ab1-ffaa-4988-9e62-bf39ec0e5a2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f084d2dae774495a8eb5ff10ab7d2f9', 'request_id': 'req-7f2e6ab1-ffaa-4988-9e62-bf39ec0e5a2c'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:58.601269 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-7f2e6ab1-ffaa-4988-9e62-bf39ec0e5a2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f084d2dae774495a8eb5ff10ab7d2f9', 'request_id': 'req-7f2e6ab1-ffaa-4988-9e62-bf39ec0e5a2c'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:58.610158 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-7f2e6ab1-ffaa-4988-9e62-bf39ec0e5a2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f084d2dae774495a8eb5ff10ab7d2f9', 'request_id': 'req-7f2e6ab1-ffaa-4988-9e62-bf39ec0e5a2c'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:58.620756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-7f2e6ab1-ffaa-4988-9e62-bf39ec0e5a2c admin admin] The token cache is being invalidate because project 9f084d2dae774495a8eb5ff10ab7d2f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:28:58.621042 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-7f2e6ab1-ffaa-4988-9e62-bf39ec0e5a2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:58.621669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-7f2e6ab1-ffaa-4988-9e62-bf39ec0e5a2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:28:58.623183 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1040/2087] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:28:58 2022] DELETE /identity/v3/projects/9f084d2dae774495a8eb5ff10ab7d2f9 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:58.628281 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a5e59-60d3-419c-81f4-87e571cca316 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:58.628754 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a5e59-60d3-419c-81f4-87e571cca316 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:58.628843 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a5e59-60d3-419c-81f4-87e571cca316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:58.650414 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-5d8a5e59-60d3-419c-81f4-87e571cca316 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:58.651633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-5d8a5e59-60d3-419c-81f4-87e571cca316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:58.717162 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1048/2088] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:58.726543 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb0565-0c13-4623-9b32-18848db34ee2 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:58.726825 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb0565-0c13-4623-9b32-18848db34ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:58.727098 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb0565-0c13-4623-9b32-18848db34ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:58.752400 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-94eb0565-0c13-4623-9b32-18848db34ee2 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:58.753604 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-94eb0565-0c13-4623-9b32-18848db34ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:58.761668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1041/2089] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:58.766452 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8a5e59-60d3-419c-81f4-87e571cca316 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:58.777613 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8a5e59-60d3-419c-81f4-87e571cca316 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:58.782615 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0ce1a-712a-4b5c-b7af-332222edefbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:58.783221 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0ce1a-712a-4b5c-b7af-332222edefbd admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:58.783489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0ce1a-712a-4b5c-b7af-332222edefbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:58.783767 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0ce1a-712a-4b5c-b7af-332222edefbd admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:58.784426 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0ce1a-712a-4b5c-b7af-332222edefbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:28:58.784724 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0ce1a-712a-4b5c-b7af-332222edefbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:58.786317 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0ce1a-712a-4b5c-b7af-332222edefbd admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:58.798175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1049/2090] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:28:58.803246 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb0565-0c13-4623-9b32-18848db34ee2 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:58.816199 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb0565-0c13-4623-9b32-18848db34ee2 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:58.822107 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24601f32-49c0-4ced-a5a2-97896f30aa75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:58.822719 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24601f32-49c0-4ced-a5a2-97896f30aa75 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:58.823000 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24601f32-49c0-4ced-a5a2-97896f30aa75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:58.823272 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24601f32-49c0-4ced-a5a2-97896f30aa75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:58.823892 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24601f32-49c0-4ced-a5a2-97896f30aa75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:58.825874 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24601f32-49c0-4ced-a5a2-97896f30aa75 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:58.859344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1042/2091] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:28:58 2022] POST /identity/v3/projects => generated 407 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:28:58.866368 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0ce1a-712a-4b5c-b7af-332222edefbd admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:58.880705 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0ce1a-712a-4b5c-b7af-332222edefbd admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:58.937968 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a2f6c-64f2-46a7-a7ba-ce3ed5769dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:58.938666 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a2f6c-64f2-46a7-a7ba-ce3ed5769dcc admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:58.938957 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a2f6c-64f2-46a7-a7ba-ce3ed5769dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:58.939213 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a2f6c-64f2-46a7-a7ba-ce3ed5769dcc admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:58.939828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a2f6c-64f2-46a7-a7ba-ce3ed5769dcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:58.941764 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a2f6c-64f2-46a7-a7ba-ce3ed5769dcc admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:58.964331 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1050/2092] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:28:58 2022] POST /identity/v3/users => generated 304 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:28:58.969100 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24601f32-49c0-4ced-a5a2-97896f30aa75 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:58.980299 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24601f32-49c0-4ced-a5a2-97896f30aa75 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:58.985324 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1ef73-8eca-41d2-b943-e94292d395f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:58.986000 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1ef73-8eca-41d2-b943-e94292d395f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:58.986276 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1ef73-8eca-41d2-b943-e94292d395f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:58.986544 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1ef73-8eca-41d2-b943-e94292d395f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:58.987257 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f1ef73-8eca-41d2-b943-e94292d395f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:58.988939 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f1ef73-8eca-41d2-b943-e94292d395f9 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:58.999828 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1043/2093] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:59.003193 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a2f6c-64f2-46a7-a7ba-ce3ed5769dcc admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:59.013851 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a2f6c-64f2-46a7-a7ba-ce3ed5769dcc admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:59.019505 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e91468-a770-41cc-a010-87743c3e4833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:59.020148 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e91468-a770-41cc-a010-87743c3e4833 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:59.020451 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e91468-a770-41cc-a010-87743c3e4833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:59.020746 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e91468-a770-41cc-a010-87743c3e4833 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:59.021396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e91468-a770-41cc-a010-87743c3e4833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:59.023128 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e91468-a770-41cc-a010-87743c3e4833 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:59.034724 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1051/2094] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:59.039257 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1ef73-8eca-41d2-b943-e94292d395f9 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:59.049959 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1ef73-8eca-41d2-b943-e94292d395f9 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:59.055358 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cba0a2-7034-41c0-93bc-3452365c1cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:59.056075 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cba0a2-7034-41c0-93bc-3452365c1cb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:59.056381 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cba0a2-7034-41c0-93bc-3452365c1cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:59.056652 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cba0a2-7034-41c0-93bc-3452365c1cb8 admin admin] PATH_INFO: `/v3/projects/6e053c96942142cb8cc0390df56872c1/users/20440d7e01d04356871980e406b70ffe/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:59.072245 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cba0a2-7034-41c0-93bc-3452365c1cb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e053c96942142cb8cc0390df56872c1, user_id=20440d7e01d04356871980e406b70ffe, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:59.074906 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cba0a2-7034-41c0-93bc-3452365c1cb8 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:59.090836 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1044/2095] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:28:59 2022] PUT /identity/v3/projects/6e053c96942142cb8cc0390df56872c1/users/20440d7e01d04356871980e406b70ffe/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:59.103140 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3d7b2-71de-4bf1-9eb2-10b851b2f3e0 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:59.103431 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3d7b2-71de-4bf1-9eb2-10b851b2f3e0 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:59.103680 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3d7b2-71de-4bf1-9eb2-10b851b2f3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:59.131273 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-42c3d7b2-71de-4bf1-9eb2-10b851b2f3e0 None None] MFA Rules not processed for user `20440d7e01d04356871980e406b70ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:59.132503 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-42c3d7b2-71de-4bf1-9eb2-10b851b2f3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:59.144377 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:59.202873 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1052/2096] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:59 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:59.207678 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cba0a2-7034-41c0-93bc-3452365c1cb8 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:59.274403 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cba0a2-7034-41c0-93bc-3452365c1cb8 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:59.279032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63ab08-684c-4efc-bc22-9df6199004c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:59.279640 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63ab08-684c-4efc-bc22-9df6199004c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:59.280054 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63ab08-684c-4efc-bc22-9df6199004c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:59.280219 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63ab08-684c-4efc-bc22-9df6199004c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:59.280847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a63ab08-684c-4efc-bc22-9df6199004c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:59.282828 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a63ab08-684c-4efc-bc22-9df6199004c7 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:59.311304 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1045/2097] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:28:59 2022] POST /identity/v3/projects => generated 409 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:28:59.317174 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3d7b2-71de-4bf1-9eb2-10b851b2f3e0 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:59.328756 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3d7b2-71de-4bf1-9eb2-10b851b2f3e0 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:59.395906 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c82740-7396-474c-a51f-61553a9822ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:59.396580 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c82740-7396-474c-a51f-61553a9822ed admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:59.396887 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c82740-7396-474c-a51f-61553a9822ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:59.397174 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c82740-7396-474c-a51f-61553a9822ed admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:59.397823 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c82740-7396-474c-a51f-61553a9822ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:59.399980 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c82740-7396-474c-a51f-61553a9822ed admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:59.424820 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1053/2098] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:28:59 2022] POST /identity/v3/users => generated 311 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:28:59.429057 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63ab08-684c-4efc-bc22-9df6199004c7 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:59.439495 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63ab08-684c-4efc-bc22-9df6199004c7 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:59.444543 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e794984-a1fd-4b0d-86da-9c5903df73ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:59.445158 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e794984-a1fd-4b0d-86da-9c5903df73ca admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:59.445434 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e794984-a1fd-4b0d-86da-9c5903df73ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:59.445701 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e794984-a1fd-4b0d-86da-9c5903df73ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:59.446333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e794984-a1fd-4b0d-86da-9c5903df73ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:59.447953 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e794984-a1fd-4b0d-86da-9c5903df73ca admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:59.462402 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1046/2099] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:28:59.466679 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c82740-7396-474c-a51f-61553a9822ed admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:59.477213 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c82740-7396-474c-a51f-61553a9822ed admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:59.481898 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46e22f-4e6a-46db-878f-2bd317f7ac46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:59.482558 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46e22f-4e6a-46db-878f-2bd317f7ac46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:59.482832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46e22f-4e6a-46db-878f-2bd317f7ac46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:59.483102 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46e22f-4e6a-46db-878f-2bd317f7ac46 admin admin] PATH_INFO: `/v3/projects/df8a7582960b4c5885d01124876430eb/users/8ed4d387a7f44e98b78cbd0852e8434e/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:59.499195 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d46e22f-4e6a-46db-878f-2bd317f7ac46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df8a7582960b4c5885d01124876430eb, user_id=8ed4d387a7f44e98b78cbd0852e8434e, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:59.502250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d46e22f-4e6a-46db-878f-2bd317f7ac46 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:59.516396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1054/2100] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:28:59 2022] PUT /identity/v3/projects/df8a7582960b4c5885d01124876430eb/users/8ed4d387a7f44e98b78cbd0852e8434e/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:59.520399 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e794984-a1fd-4b0d-86da-9c5903df73ca admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:59.530961 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e794984-a1fd-4b0d-86da-9c5903df73ca admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:59.546836 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:28:59.612842 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751d8ee-87ec-4063-94e4-ad817c633c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:59.613453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751d8ee-87ec-4063-94e4-ad817c633c6a admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:59.613738 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751d8ee-87ec-4063-94e4-ad817c633c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:59.614013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751d8ee-87ec-4063-94e4-ad817c633c6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:59.614684 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751d8ee-87ec-4063-94e4-ad817c633c6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:59.616444 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751d8ee-87ec-4063-94e4-ad817c633c6a admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:59.626708 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1047/2101] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:28:59.630880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46e22f-4e6a-46db-878f-2bd317f7ac46 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:59.641714 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46e22f-4e6a-46db-878f-2bd317f7ac46 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:59.646601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3fcfb4-cf72-4d6a-8e3b-50c0f2e02d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:59.647240 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fcfb4-cf72-4d6a-8e3b-50c0f2e02d1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:59.647517 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fcfb4-cf72-4d6a-8e3b-50c0f2e02d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:59.647789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fcfb4-cf72-4d6a-8e3b-50c0f2e02d1e admin admin] PATH_INFO: `/v3/projects/df8a7582960b4c5885d01124876430eb/users/8ed4d387a7f44e98b78cbd0852e8434e/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:59.649769 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3fcfb4-cf72-4d6a-8e3b-50c0f2e02d1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=df8a7582960b4c5885d01124876430eb, user_id=8ed4d387a7f44e98b78cbd0852e8434e, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:59.652461 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3fcfb4-cf72-4d6a-8e3b-50c0f2e02d1e admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:59.662944 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1055/2102] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:28:59 2022] PUT /identity/v3/projects/df8a7582960b4c5885d01124876430eb/users/8ed4d387a7f44e98b78cbd0852e8434e/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:28:59.673378 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bf82b-cf80-4bf1-93bd-d8fe165d90e9 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:59.673664 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bf82b-cf80-4bf1-93bd-d8fe165d90e9 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:59.673939 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bf82b-cf80-4bf1-93bd-d8fe165d90e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:59.702057 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-4d3bf82b-cf80-4bf1-93bd-d8fe165d90e9 None None] MFA Rules not processed for user `8ed4d387a7f44e98b78cbd0852e8434e`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:28:59.703263 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-4d3bf82b-cf80-4bf1-93bd-d8fe165d90e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:59.781968 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1048/2103] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:28:59 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:28:59.793143 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3fcfb4-cf72-4d6a-8e3b-50c0f2e02d1e admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:28:59.795254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-5f3fcfb4-cf72-4d6a-8e3b-50c0f2e02d1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:28:59.860671 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3fcfb4-cf72-4d6a-8e3b-50c0f2e02d1e admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:28:59.864881 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c0c05-b8ed-4116-b8df-0723a61d23d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:28:59.865493 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c0c05-b8ed-4116-b8df-0723a61d23d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:28:59.865772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c0c05-b8ed-4116-b8df-0723a61d23d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:28:59.866039 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c0c05-b8ed-4116-b8df-0723a61d23d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:28:59.876070 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33c0c05-b8ed-4116-b8df-0723a61d23d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:28:59.877534 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33c0c05-b8ed-4116-b8df-0723a61d23d3 service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:28:59.887381 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1056/2104] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:28:59 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:29:00.050085 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bf82b-cf80-4bf1-93bd-d8fe165d90e9 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:00.059972 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bf82b-cf80-4bf1-93bd-d8fe165d90e9 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:00.063802 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54f63f-ed0b-4dbe-9cea-8585a313b3c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:00.064494 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54f63f-ed0b-4dbe-9cea-8585a313b3c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:00.064772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54f63f-ed0b-4dbe-9cea-8585a313b3c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:00.065040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54f63f-ed0b-4dbe-9cea-8585a313b3c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:00.141655 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce54f63f-ed0b-4dbe-9cea-8585a313b3c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:00.143109 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce54f63f-ed0b-4dbe-9cea-8585a313b3c3 service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:00.152552 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1049/2105] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:29:00 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:29:01.987580 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d48d9-2d33-424c-9541-d150fad6eb6b None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:01.988076 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d48d9-2d33-424c-9541-d150fad6eb6b None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:01.988164 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d48d9-2d33-424c-9541-d150fad6eb6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:02.007367 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-144d48d9-2d33-424c-9541-d150fad6eb6b None None] MFA Rules not processed for user `d3538271ebfd41aabe9f3da85feb7bca`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:02.008583 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-144d48d9-2d33-424c-9541-d150fad6eb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:02.012917 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:29:02.070641 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1057/2106] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Aug 23 04:29:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:29:02.117689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54f63f-ed0b-4dbe-9cea-8585a313b3c3 service cinder] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:02.120124 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-ce54f63f-ed0b-4dbe-9cea-8585a313b3c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:02.159646 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:29:02.198243 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54f63f-ed0b-4dbe-9cea-8585a313b3c3 service cinder] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:02.203882 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402831b-9f34-4d6d-baaf-3559bbb7d1d3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f1418fd211a448da1a50175dce13b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:02.204517 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402831b-9f34-4d6d-baaf-3559bbb7d1d3 service swift] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:02.204798 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402831b-9f34-4d6d-baaf-3559bbb7d1d3 service swift] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:02.205070 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402831b-9f34-4d6d-baaf-3559bbb7d1d3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:02.213391 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e402831b-9f34-4d6d-baaf-3559bbb7d1d3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:02.214984 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e402831b-9f34-4d6d-baaf-3559bbb7d1d3 service swift] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:02.223983 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1050/2107] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Aug 23 04:29:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 04:29:02.804997 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7addd-878b-40b7-ab5d-faf309b7c66e None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:02.805281 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7addd-878b-40b7-ab5d-faf309b7c66e None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:02.805554 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7addd-878b-40b7-ab5d-faf309b7c66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:02.825458 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-97a7addd-878b-40b7-ab5d-faf309b7c66e None None] MFA Rules not processed for user `d3538271ebfd41aabe9f3da85feb7bca`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:02.826714 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-97a7addd-878b-40b7-ab5d-faf309b7c66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:02.834191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1058/2108] 10.222.0.18 () {58 vars in 967 bytes} [Tue Aug 23 04:29:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:29:02.848129 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402831b-9f34-4d6d-baaf-3559bbb7d1d3 service swift] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:02.858358 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402831b-9f34-4d6d-baaf-3559bbb7d1d3 service swift] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:02.865368 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd36f4-8a35-498c-8505-1cfaa26e1d0e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f1418fd211a448da1a50175dce13b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:02.866079 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd36f4-8a35-498c-8505-1cfaa26e1d0e service swift] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:02.866454 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd36f4-8a35-498c-8505-1cfaa26e1d0e service swift] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:02.866745 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd36f4-8a35-498c-8505-1cfaa26e1d0e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:02.875517 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd36f4-8a35-498c-8505-1cfaa26e1d0e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:02.877376 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd36f4-8a35-498c-8505-1cfaa26e1d0e service swift] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:02.887264 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1051/2109] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Aug 23 04:29:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 04:29:03.153028 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b41dd5-2a0b-46cb-9976-83efebc9dd83 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:03.153516 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b41dd5-2a0b-46cb-9976-83efebc9dd83 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:03.153617 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b41dd5-2a0b-46cb-9976-83efebc9dd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:03.173918 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-00b41dd5-2a0b-46cb-9976-83efebc9dd83 None None] MFA Rules not processed for user `d3538271ebfd41aabe9f3da85feb7bca`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:03.175145 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-00b41dd5-2a0b-46cb-9976-83efebc9dd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:03.182825 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1059/2110] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Aug 23 04:29:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:29:03.198248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd36f4-8a35-498c-8505-1cfaa26e1d0e service swift] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:03.211407 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd36f4-8a35-498c-8505-1cfaa26e1d0e service swift] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:03.218232 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbcf2fb-1fdc-478d-bd19-f06e6142880d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f1418fd211a448da1a50175dce13b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:03.218839 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbcf2fb-1fdc-478d-bd19-f06e6142880d service swift] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:03.219124 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbcf2fb-1fdc-478d-bd19-f06e6142880d service swift] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:03.219374 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbcf2fb-1fdc-478d-bd19-f06e6142880d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:03.229001 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbcf2fb-1fdc-478d-bd19-f06e6142880d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:03.230624 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbcf2fb-1fdc-478d-bd19-f06e6142880d service swift] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:03.239344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1052/2111] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Aug 23 04:29:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 04:29:03.596130 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3523257-97c0-482f-8cb4-724bcdc053c0 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:03.596502 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3523257-97c0-482f-8cb4-724bcdc053c0 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:03.596772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3523257-97c0-482f-8cb4-724bcdc053c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:03.616757 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-b3523257-97c0-482f-8cb4-724bcdc053c0 None None] MFA Rules not processed for user `d3538271ebfd41aabe9f3da85feb7bca`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:03.617943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-b3523257-97c0-482f-8cb4-724bcdc053c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:03.626114 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1060/2112] 10.222.0.18 () {58 vars in 967 bytes} [Tue Aug 23 04:29:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:29:03.640708 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbcf2fb-1fdc-478d-bd19-f06e6142880d service swift] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:03.651744 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbcf2fb-1fdc-478d-bd19-f06e6142880d service swift] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:03.657832 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602cb524-8b1d-494c-bc4e-80cce5699628 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f1418fd211a448da1a50175dce13b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:03.658419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602cb524-8b1d-494c-bc4e-80cce5699628 service swift] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:03.658699 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602cb524-8b1d-494c-bc4e-80cce5699628 service swift] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:03.658952 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602cb524-8b1d-494c-bc4e-80cce5699628 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:03.667352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602cb524-8b1d-494c-bc4e-80cce5699628 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:03.669219 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602cb524-8b1d-494c-bc4e-80cce5699628 service swift] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:03.677620 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1053/2113] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Aug 23 04:29:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 04:29:05.821721 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3523257-97c0-482f-8cb4-724bcdc053c0 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:05.823463 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-b3523257-97c0-482f-8cb4-724bcdc053c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:05.900388 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3523257-97c0-482f-8cb4-724bcdc053c0 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:05.908725 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be473df0-ced3-48c3-a72a-967e9c7b0875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:05.909350 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be473df0-ced3-48c3-a72a-967e9c7b0875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:05.909657 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be473df0-ced3-48c3-a72a-967e9c7b0875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:05.909915 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be473df0-ced3-48c3-a72a-967e9c7b0875 admin admin] PATH_INFO: `/v3/users/9cd4aa1f6a4141289a6ae570fb4db6f1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:05.911152 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be473df0-ced3-48c3-a72a-967e9c7b0875 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cd4aa1f6a4141289a6ae570fb4db6f1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:05.913211 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be473df0-ced3-48c3-a72a-967e9c7b0875 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:05.986050 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-be473df0-ced3-48c3-a72a-967e9c7b0875 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cd4aa1f6a4141289a6ae570fb4db6f1'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:05.986804 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-be473df0-ced3-48c3-a72a-967e9c7b0875 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cd4aa1f6a4141289a6ae570fb4db6f1'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:06.003626 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-be473df0-ced3-48c3-a72a-967e9c7b0875 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cd4aa1f6a4141289a6ae570fb4db6f1'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:06.018062 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-be473df0-ced3-48c3-a72a-967e9c7b0875 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cd4aa1f6a4141289a6ae570fb4db6f1'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:06.025128 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-be473df0-ced3-48c3-a72a-967e9c7b0875 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cd4aa1f6a4141289a6ae570fb4db6f1'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:06.032464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1061/2114] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:29:05 2022] DELETE /identity/v3/users/9cd4aa1f6a4141289a6ae570fb4db6f1 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:29:06.036946 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5daba-f9a5-426d-b2bb-01d2a796ce4f None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:06.037382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5daba-f9a5-426d-b2bb-01d2a796ce4f None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:06.037506 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5daba-f9a5-426d-b2bb-01d2a796ce4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:06.057461 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-27f5daba-f9a5-426d-b2bb-01d2a796ce4f None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:06.058781 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-27f5daba-f9a5-426d-b2bb-01d2a796ce4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:06.127808 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1054/2115] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:29:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:29:06.137923 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be473df0-ced3-48c3-a72a-967e9c7b0875 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:06.139343 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-be473df0-ced3-48c3-a72a-967e9c7b0875 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:06.154316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:29:06.200895 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be473df0-ced3-48c3-a72a-967e9c7b0875 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:06.204666 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe612549-9dd0-46cf-a76a-1d463dd7bbcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:06.205280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe612549-9dd0-46cf-a76a-1d463dd7bbcb service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:06.205559 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe612549-9dd0-46cf-a76a-1d463dd7bbcb service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:06.205837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe612549-9dd0-46cf-a76a-1d463dd7bbcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:06.214969 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe612549-9dd0-46cf-a76a-1d463dd7bbcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:06.216494 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe612549-9dd0-46cf-a76a-1d463dd7bbcb service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:06.231472 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1062/2116] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:29:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:29:06.420879 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5daba-f9a5-426d-b2bb-01d2a796ce4f None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:06.422638 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-27f5daba-f9a5-426d-b2bb-01d2a796ce4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:06.450847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5daba-f9a5-426d-b2bb-01d2a796ce4f None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:06.464287 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ecc7c4-47e7-4ef6-8a69-8c882a2009b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:06.464961 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ecc7c4-47e7-4ef6-8a69-8c882a2009b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:06.465238 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ecc7c4-47e7-4ef6-8a69-8c882a2009b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:06.465518 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ecc7c4-47e7-4ef6-8a69-8c882a2009b2 admin admin] PATH_INFO: `/v3/projects/aec8505d96be450188e176d091286fc4` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:06.466651 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ecc7c4-47e7-4ef6-8a69-8c882a2009b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aec8505d96be450188e176d091286fc4)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:06.468606 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ecc7c4-47e7-4ef6-8a69-8c882a2009b2 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:06.531500 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-24ecc7c4-47e7-4ef6-8a69-8c882a2009b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aec8505d96be450188e176d091286fc4', 'request_id': 'req-24ecc7c4-47e7-4ef6-8a69-8c882a2009b2'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:06.538410 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-24ecc7c4-47e7-4ef6-8a69-8c882a2009b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aec8505d96be450188e176d091286fc4', 'request_id': 'req-24ecc7c4-47e7-4ef6-8a69-8c882a2009b2'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:06.547082 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-24ecc7c4-47e7-4ef6-8a69-8c882a2009b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aec8505d96be450188e176d091286fc4', 'request_id': 'req-24ecc7c4-47e7-4ef6-8a69-8c882a2009b2'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:06.560070 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-24ecc7c4-47e7-4ef6-8a69-8c882a2009b2 admin admin] The token cache is being invalidate because project aec8505d96be450188e176d091286fc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:29:06.560361 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-24ecc7c4-47e7-4ef6-8a69-8c882a2009b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:06.561124 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-24ecc7c4-47e7-4ef6-8a69-8c882a2009b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:06.562776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1055/2117] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:29:06 2022] DELETE /identity/v3/projects/aec8505d96be450188e176d091286fc4 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:29:08.660283 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe612549-9dd0-46cf-a76a-1d463dd7bbcb service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:08.661945 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-fe612549-9dd0-46cf-a76a-1d463dd7bbcb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:08.675301 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe612549-9dd0-46cf-a76a-1d463dd7bbcb service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:08.735570 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa282f7-5cdb-48fa-bedc-504ca6a8fb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:08.736458 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa282f7-5cdb-48fa-bedc-504ca6a8fb8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:08.736458 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa282f7-5cdb-48fa-bedc-504ca6a8fb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:08.736915 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa282f7-5cdb-48fa-bedc-504ca6a8fb8f admin admin] PATH_INFO: `/v3/users/20440d7e01d04356871980e406b70ffe` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:08.738050 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa282f7-5cdb-48fa-bedc-504ca6a8fb8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=20440d7e01d04356871980e406b70ffe)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:08.740024 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa282f7-5cdb-48fa-bedc-504ca6a8fb8f admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:08.799684 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-1fa282f7-5cdb-48fa-bedc-504ca6a8fb8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20440d7e01d04356871980e406b70ffe'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:08.800835 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-1fa282f7-5cdb-48fa-bedc-504ca6a8fb8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20440d7e01d04356871980e406b70ffe'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:08.809739 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-1fa282f7-5cdb-48fa-bedc-504ca6a8fb8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20440d7e01d04356871980e406b70ffe'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:08.822324 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-1fa282f7-5cdb-48fa-bedc-504ca6a8fb8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20440d7e01d04356871980e406b70ffe'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:08.832407 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-1fa282f7-5cdb-48fa-bedc-504ca6a8fb8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20440d7e01d04356871980e406b70ffe'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:08.838236 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1063/2118] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:29:08 2022] DELETE /identity/v3/users/20440d7e01d04356871980e406b70ffe => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:29:08.842778 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394cbb4-0381-4c71-a77c-98027479dbf2 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:08.843286 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394cbb4-0381-4c71-a77c-98027479dbf2 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:08.843286 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394cbb4-0381-4c71-a77c-98027479dbf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:08.858256 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:29:08.864487 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-e394cbb4-0381-4c71-a77c-98027479dbf2 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:08.866453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-e394cbb4-0381-4c71-a77c-98027479dbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:08.931025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1056/2119] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:29:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:29:08.941092 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa282f7-5cdb-48fa-bedc-504ca6a8fb8f admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:08.942761 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-1fa282f7-5cdb-48fa-bedc-504ca6a8fb8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:08.984799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:29:09.012086 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa282f7-5cdb-48fa-bedc-504ca6a8fb8f admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:09.017228 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c5161-86f7-453d-872c-a62c1a6717b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:09.017836 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c5161-86f7-453d-872c-a62c1a6717b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:09.018117 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c5161-86f7-453d-872c-a62c1a6717b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:09.018388 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c5161-86f7-453d-872c-a62c1a6717b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:09.028267 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c5161-86f7-453d-872c-a62c1a6717b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:09.029714 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c5161-86f7-453d-872c-a62c1a6717b8 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:09.039970 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1064/2120] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:29:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:29:09.344139 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394cbb4-0381-4c71-a77c-98027479dbf2 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:09.346302 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-e394cbb4-0381-4c71-a77c-98027479dbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:09.361176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394cbb4-0381-4c71-a77c-98027479dbf2 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:09.367399 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dcf7b0-9ad0-4f16-8595-c6190cfd3233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:09.368392 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dcf7b0-9ad0-4f16-8595-c6190cfd3233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:09.368655 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dcf7b0-9ad0-4f16-8595-c6190cfd3233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:09.368928 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dcf7b0-9ad0-4f16-8595-c6190cfd3233 admin admin] PATH_INFO: `/v3/projects/6e053c96942142cb8cc0390df56872c1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:09.370228 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dcf7b0-9ad0-4f16-8595-c6190cfd3233 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e053c96942142cb8cc0390df56872c1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:09.372135 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dcf7b0-9ad0-4f16-8595-c6190cfd3233 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:09.429655 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-68dcf7b0-9ad0-4f16-8595-c6190cfd3233 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e053c96942142cb8cc0390df56872c1', 'request_id': 'req-68dcf7b0-9ad0-4f16-8595-c6190cfd3233'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:09.435847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-68dcf7b0-9ad0-4f16-8595-c6190cfd3233 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e053c96942142cb8cc0390df56872c1', 'request_id': 'req-68dcf7b0-9ad0-4f16-8595-c6190cfd3233'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:09.444802 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-68dcf7b0-9ad0-4f16-8595-c6190cfd3233 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e053c96942142cb8cc0390df56872c1', 'request_id': 'req-68dcf7b0-9ad0-4f16-8595-c6190cfd3233'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:09.456649 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-68dcf7b0-9ad0-4f16-8595-c6190cfd3233 admin admin] The token cache is being invalidate because project 6e053c96942142cb8cc0390df56872c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:29:09.456865 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-68dcf7b0-9ad0-4f16-8595-c6190cfd3233 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:09.462389 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-68dcf7b0-9ad0-4f16-8595-c6190cfd3233 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:09.462389 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1057/2121] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:29:09 2022] DELETE /identity/v3/projects/6e053c96942142cb8cc0390df56872c1 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:29:09.463435 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c5161-86f7-453d-872c-a62c1a6717b8 service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:09.474415 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c5161-86f7-453d-872c-a62c1a6717b8 service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:09.535957 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c92fd-b0ea-4f1f-a0db-15d6e24933ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:09.536592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c92fd-b0ea-4f1f-a0db-15d6e24933ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:09.536867 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c92fd-b0ea-4f1f-a0db-15d6e24933ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:09.537140 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c92fd-b0ea-4f1f-a0db-15d6e24933ae admin admin] PATH_INFO: `/v3/users/8ed4d387a7f44e98b78cbd0852e8434e` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:09.538232 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c92fd-b0ea-4f1f-a0db-15d6e24933ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ed4d387a7f44e98b78cbd0852e8434e)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:09.540210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c92fd-b0ea-4f1f-a0db-15d6e24933ae admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:09.589793 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-764c92fd-b0ea-4f1f-a0db-15d6e24933ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ed4d387a7f44e98b78cbd0852e8434e'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:09.590364 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-764c92fd-b0ea-4f1f-a0db-15d6e24933ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ed4d387a7f44e98b78cbd0852e8434e'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:09.600703 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-764c92fd-b0ea-4f1f-a0db-15d6e24933ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ed4d387a7f44e98b78cbd0852e8434e'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:09.612501 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-764c92fd-b0ea-4f1f-a0db-15d6e24933ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ed4d387a7f44e98b78cbd0852e8434e'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:09.619159 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-764c92fd-b0ea-4f1f-a0db-15d6e24933ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ed4d387a7f44e98b78cbd0852e8434e'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:09.625265 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1065/2122] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:29:09 2022] DELETE /identity/v3/users/8ed4d387a7f44e98b78cbd0852e8434e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:29:09.973365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dcf7b0-9ad0-4f16-8595-c6190cfd3233 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:09.974887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-68dcf7b0-9ad0-4f16-8595-c6190cfd3233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:09.988362 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dcf7b0-9ad0-4f16-8595-c6190cfd3233 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:10.054138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a5b01-3664-448c-9136-faf62ff11bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:10.054796 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a5b01-3664-448c-9136-faf62ff11bc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:10.055071 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a5b01-3664-448c-9136-faf62ff11bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:10.055339 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a5b01-3664-448c-9136-faf62ff11bc4 admin admin] PATH_INFO: `/v3/projects/df8a7582960b4c5885d01124876430eb` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:10.056533 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30a5b01-3664-448c-9136-faf62ff11bc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df8a7582960b4c5885d01124876430eb)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:10.058451 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30a5b01-3664-448c-9136-faf62ff11bc4 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:10.093881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:29:10.107634 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a30a5b01-3664-448c-9136-faf62ff11bc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df8a7582960b4c5885d01124876430eb', 'request_id': 'req-a30a5b01-3664-448c-9136-faf62ff11bc4'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:10.113988 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a30a5b01-3664-448c-9136-faf62ff11bc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df8a7582960b4c5885d01124876430eb', 'request_id': 'req-a30a5b01-3664-448c-9136-faf62ff11bc4'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:10.122276 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a30a5b01-3664-448c-9136-faf62ff11bc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df8a7582960b4c5885d01124876430eb', 'request_id': 'req-a30a5b01-3664-448c-9136-faf62ff11bc4'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:10.137699 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a30a5b01-3664-448c-9136-faf62ff11bc4 admin admin] The token cache is being invalidate because project df8a7582960b4c5885d01124876430eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:29:10.137989 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a30a5b01-3664-448c-9136-faf62ff11bc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:10.138668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a30a5b01-3664-448c-9136-faf62ff11bc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:10.140243 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1058/2123] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:29:09 2022] DELETE /identity/v3/projects/df8a7582960b4c5885d01124876430eb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:29:10.146635 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5df63-8172-4a4f-918a-aa219ebf8b36 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:10.147106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5df63-8172-4a4f-918a-aa219ebf8b36 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:10.147219 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5df63-8172-4a4f-918a-aa219ebf8b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:10.170562 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-19a5df63-8172-4a4f-918a-aa219ebf8b36 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:10.171755 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-19a5df63-8172-4a4f-918a-aa219ebf8b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:10.226154 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:29:10.248493 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1066/2124] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:29:10.250428 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8161a-eb0f-4c71-a7d1-d12301268fc0 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:10.251311 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8161a-eb0f-4c71-a7d1-d12301268fc0 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:10.251591 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8161a-eb0f-4c71-a7d1-d12301268fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:10.272516 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-69a8161a-eb0f-4c71-a7d1-d12301268fc0 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:10.273733 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-69a8161a-eb0f-4c71-a7d1-d12301268fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:10.284594 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1059/2125] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:29:10.289260 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5df63-8172-4a4f-918a-aa219ebf8b36 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:10.300321 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5df63-8172-4a4f-918a-aa219ebf8b36 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:10.308603 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412cf741-41d2-4015-a8be-295269dee5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:10.309239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412cf741-41d2-4015-a8be-295269dee5b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:10.309528 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412cf741-41d2-4015-a8be-295269dee5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:10.309873 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412cf741-41d2-4015-a8be-295269dee5b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:10.310814 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412cf741-41d2-4015-a8be-295269dee5b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:29:10.311244 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412cf741-41d2-4015-a8be-295269dee5b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:10.313471 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412cf741-41d2-4015-a8be-295269dee5b6 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:10.325486 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1067/2126] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:29:10.330374 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a8161a-eb0f-4c71-a7d1-d12301268fc0 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:10.339647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a8161a-eb0f-4c71-a7d1-d12301268fc0 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:10.345246 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf44f4-0e7b-4341-a3ec-b1afd0564b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:10.345846 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf44f4-0e7b-4341-a3ec-b1afd0564b19 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:10.346122 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf44f4-0e7b-4341-a3ec-b1afd0564b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:10.346394 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf44f4-0e7b-4341-a3ec-b1afd0564b19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:10.347009 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf44f4-0e7b-4341-a3ec-b1afd0564b19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:10.349327 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf44f4-0e7b-4341-a3ec-b1afd0564b19 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:10.379075 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1060/2127] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:29:10 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:29:10.383417 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412cf741-41d2-4015-a8be-295269dee5b6 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:10.393747 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412cf741-41d2-4015-a8be-295269dee5b6 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:10.452777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edcd817-2f48-4d37-a12c-b805b33bcd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:10.453507 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcd817-2f48-4d37-a12c-b805b33bcd40 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:10.453854 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcd817-2f48-4d37-a12c-b805b33bcd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:10.454205 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcd817-2f48-4d37-a12c-b805b33bcd40 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:10.454849 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edcd817-2f48-4d37-a12c-b805b33bcd40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:10.456883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edcd817-2f48-4d37-a12c-b805b33bcd40 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:10.480329 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1068/2128] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:29:10 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:29:10.484872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf44f4-0e7b-4341-a3ec-b1afd0564b19 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:10.496197 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf44f4-0e7b-4341-a3ec-b1afd0564b19 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:10.501252 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d0b26-08e2-4951-9a05-9bd11d467369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:10.501851 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d0b26-08e2-4951-9a05-9bd11d467369 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:10.502122 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d0b26-08e2-4951-9a05-9bd11d467369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:10.502410 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d0b26-08e2-4951-9a05-9bd11d467369 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:10.503050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d0b26-08e2-4951-9a05-9bd11d467369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:10.504689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d0b26-08e2-4951-9a05-9bd11d467369 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:10.515373 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1061/2129] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:29:10.520040 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edcd817-2f48-4d37-a12c-b805b33bcd40 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:10.530844 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edcd817-2f48-4d37-a12c-b805b33bcd40 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:10.536415 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718588fb-331c-482c-9e29-213d68f5eca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:10.537051 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718588fb-331c-482c-9e29-213d68f5eca1 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:10.537549 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718588fb-331c-482c-9e29-213d68f5eca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:10.537657 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718588fb-331c-482c-9e29-213d68f5eca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:10.538234 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718588fb-331c-482c-9e29-213d68f5eca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:10.539816 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718588fb-331c-482c-9e29-213d68f5eca1 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:10.553525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1069/2130] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:29:10.556596 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d0b26-08e2-4951-9a05-9bd11d467369 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:10.567489 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d0b26-08e2-4951-9a05-9bd11d467369 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:10.572506 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91523e-c6fe-4cd4-9b5b-9fb41e0820d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:10.572944 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91523e-c6fe-4cd4-9b5b-9fb41e0820d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:10.573222 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91523e-c6fe-4cd4-9b5b-9fb41e0820d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:10.573492 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91523e-c6fe-4cd4-9b5b-9fb41e0820d9 admin admin] PATH_INFO: `/v3/projects/78f8cb4784554c44aa3a89a9014c7a6d/users/79881618a76a40c6884aada4a6d10f42/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:10.588586 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c91523e-c6fe-4cd4-9b5b-9fb41e0820d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78f8cb4784554c44aa3a89a9014c7a6d, user_id=79881618a76a40c6884aada4a6d10f42, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:10.591384 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c91523e-c6fe-4cd4-9b5b-9fb41e0820d9 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:10.605578 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1062/2131] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:29:10 2022] PUT /identity/v3/projects/78f8cb4784554c44aa3a89a9014c7a6d/users/79881618a76a40c6884aada4a6d10f42/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:29:10.616101 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b208496-e5bb-4aea-a773-51bff12d698c None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:10.616390 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b208496-e5bb-4aea-a773-51bff12d698c None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:10.616664 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b208496-e5bb-4aea-a773-51bff12d698c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:10.645303 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-7b208496-e5bb-4aea-a773-51bff12d698c None None] MFA Rules not processed for user `79881618a76a40c6884aada4a6d10f42`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:10.646522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-7b208496-e5bb-4aea-a773-51bff12d698c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:10.719757 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1070/2132] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:29:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:29:10.724765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91523e-c6fe-4cd4-9b5b-9fb41e0820d9 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:10.793972 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91523e-c6fe-4cd4-9b5b-9fb41e0820d9 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:10.799258 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c1f8f-82c6-40df-90f4-b2a164c137c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:10.799870 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c1f8f-82c6-40df-90f4-b2a164c137c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:10.800185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c1f8f-82c6-40df-90f4-b2a164c137c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:10.800457 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c1f8f-82c6-40df-90f4-b2a164c137c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:10.801144 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c1f8f-82c6-40df-90f4-b2a164c137c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:10.803067 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c1f8f-82c6-40df-90f4-b2a164c137c6 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:10.831935 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1063/2133] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:29:10 2022] POST /identity/v3/projects => generated 367 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:29:10.838450 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b208496-e5bb-4aea-a773-51bff12d698c None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:10.850568 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b208496-e5bb-4aea-a773-51bff12d698c None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:10.897417 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:29:10.907908 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bfff1c-2783-486e-bcd4-449ef7b94d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:10.908547 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfff1c-2783-486e-bcd4-449ef7b94d90 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:10.908822 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfff1c-2783-486e-bcd4-449ef7b94d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:10.909100 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfff1c-2783-486e-bcd4-449ef7b94d90 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:10.909725 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bfff1c-2783-486e-bcd4-449ef7b94d90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:10.911645 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bfff1c-2783-486e-bcd4-449ef7b94d90 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:10.932529 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1071/2134] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:29:10 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:29:10.936635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c1f8f-82c6-40df-90f4-b2a164c137c6 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:10.947442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c1f8f-82c6-40df-90f4-b2a164c137c6 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:10.952881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6dc817-5022-44c8-84ba-903824e532b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:10.953547 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dc817-5022-44c8-84ba-903824e532b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:10.953778 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dc817-5022-44c8-84ba-903824e532b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:10.954144 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dc817-5022-44c8-84ba-903824e532b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:10.954811 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6dc817-5022-44c8-84ba-903824e532b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:10.956497 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6dc817-5022-44c8-84ba-903824e532b9 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:10.967727 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1064/2135] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:29:10.972920 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bfff1c-2783-486e-bcd4-449ef7b94d90 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:10.988543 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bfff1c-2783-486e-bcd4-449ef7b94d90 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:10.997586 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442883a-b32a-41dc-aee6-8debae8f647f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:10.998580 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442883a-b32a-41dc-aee6-8debae8f647f admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:10.998961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442883a-b32a-41dc-aee6-8debae8f647f admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:10.999340 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442883a-b32a-41dc-aee6-8debae8f647f admin admin] PATH_INFO: `/v3/projects/af45a7c156b844eeb2c4ef506eb1a86e/users/8e0c161c55cd4335844aa9d4c32ff62a/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:11.016832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442883a-b32a-41dc-aee6-8debae8f647f admin admin] RBAC: Authorizing `identity:create_grant(project_id=af45a7c156b844eeb2c4ef506eb1a86e, user_id=8e0c161c55cd4335844aa9d4c32ff62a, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:11.019487 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442883a-b32a-41dc-aee6-8debae8f647f admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:11.035339 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1072/2136] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:29:10 2022] PUT /identity/v3/projects/af45a7c156b844eeb2c4ef506eb1a86e/users/8e0c161c55cd4335844aa9d4c32ff62a/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:29:11.039481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6dc817-5022-44c8-84ba-903824e532b9 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:11.051152 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6dc817-5022-44c8-84ba-903824e532b9 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:11.080931 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:29:11.112395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc424627-663a-45d0-b889-24102059afd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:11.113008 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc424627-663a-45d0-b889-24102059afd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:11.113284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc424627-663a-45d0-b889-24102059afd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:11.113698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc424627-663a-45d0-b889-24102059afd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:11.114382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc424627-663a-45d0-b889-24102059afd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:11.116025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc424627-663a-45d0-b889-24102059afd1 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:11.126789 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1065/2137] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:29:11.131088 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442883a-b32a-41dc-aee6-8debae8f647f admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:11.141679 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442883a-b32a-41dc-aee6-8debae8f647f admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:11.146427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5196b44d-39b1-48d5-a269-bcd947a7e251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:11.147123 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196b44d-39b1-48d5-a269-bcd947a7e251 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:11.147659 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196b44d-39b1-48d5-a269-bcd947a7e251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:11.147659 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196b44d-39b1-48d5-a269-bcd947a7e251 admin admin] PATH_INFO: `/v3/projects/af45a7c156b844eeb2c4ef506eb1a86e/users/8e0c161c55cd4335844aa9d4c32ff62a/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:11.149610 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5196b44d-39b1-48d5-a269-bcd947a7e251 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af45a7c156b844eeb2c4ef506eb1a86e, user_id=8e0c161c55cd4335844aa9d4c32ff62a, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:11.152292 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5196b44d-39b1-48d5-a269-bcd947a7e251 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:11.163321 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1073/2138] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:29:11 2022] PUT /identity/v3/projects/af45a7c156b844eeb2c4ef506eb1a86e/users/8e0c161c55cd4335844aa9d4c32ff62a/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:29:11.174639 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34145f-1931-4a78-9894-38222f6c57c5 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:11.174926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34145f-1931-4a78-9894-38222f6c57c5 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:11.175197 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34145f-1931-4a78-9894-38222f6c57c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:11.203899 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-4c34145f-1931-4a78-9894-38222f6c57c5 None None] MFA Rules not processed for user `8e0c161c55cd4335844aa9d4c32ff62a`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:11.205217 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-4c34145f-1931-4a78-9894-38222f6c57c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:11.283075 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1066/2139] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:29:11 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:29:11.294472 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5196b44d-39b1-48d5-a269-bcd947a7e251 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:11.296430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-5196b44d-39b1-48d5-a269-bcd947a7e251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:11.363846 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5196b44d-39b1-48d5-a269-bcd947a7e251 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:11.367678 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba70ff-7e06-4f33-baf4-1fdc7646648f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:11.368326 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba70ff-7e06-4f33-baf4-1fdc7646648f service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:11.368607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba70ff-7e06-4f33-baf4-1fdc7646648f service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:11.368875 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba70ff-7e06-4f33-baf4-1fdc7646648f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:11.378179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba70ff-7e06-4f33-baf4-1fdc7646648f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:11.379610 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba70ff-7e06-4f33-baf4-1fdc7646648f service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:11.388983 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1074/2140] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:29:11 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:29:11.484003 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34145f-1931-4a78-9894-38222f6c57c5 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:11.494088 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34145f-1931-4a78-9894-38222f6c57c5 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:11.498067 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8cd9e-b4fa-4aaa-ba91-223cff5acf52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:11.498670 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8cd9e-b4fa-4aaa-ba91-223cff5acf52 service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:11.498943 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8cd9e-b4fa-4aaa-ba91-223cff5acf52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:11.499207 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8cd9e-b4fa-4aaa-ba91-223cff5acf52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:11.567720 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a8cd9e-b4fa-4aaa-ba91-223cff5acf52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:11.569218 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a8cd9e-b4fa-4aaa-ba91-223cff5acf52 service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:11.579526 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1067/2141] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:29:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:29:18.735042 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba70ff-7e06-4f33-baf4-1fdc7646648f service cinder] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:18.807343 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba70ff-7e06-4f33-baf4-1fdc7646648f service cinder] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:18.811998 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff50cd-bcaa-42c7-8285-0dbff8e70b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:18.812621 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff50cd-bcaa-42c7-8285-0dbff8e70b21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:18.812903 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff50cd-bcaa-42c7-8285-0dbff8e70b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:18.813185 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff50cd-bcaa-42c7-8285-0dbff8e70b21 admin admin] PATH_INFO: `/v3/users/79881618a76a40c6884aada4a6d10f42` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:18.814439 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff50cd-bcaa-42c7-8285-0dbff8e70b21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79881618a76a40c6884aada4a6d10f42)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:18.816510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff50cd-bcaa-42c7-8285-0dbff8e70b21 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:18.873384 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a3ff50cd-bcaa-42c7-8285-0dbff8e70b21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79881618a76a40c6884aada4a6d10f42'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:18.874239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a3ff50cd-bcaa-42c7-8285-0dbff8e70b21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79881618a76a40c6884aada4a6d10f42'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:18.878121 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:29:18.886325 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a3ff50cd-bcaa-42c7-8285-0dbff8e70b21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79881618a76a40c6884aada4a6d10f42'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:18.902281 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a3ff50cd-bcaa-42c7-8285-0dbff8e70b21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79881618a76a40c6884aada4a6d10f42'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:18.910242 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a3ff50cd-bcaa-42c7-8285-0dbff8e70b21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79881618a76a40c6884aada4a6d10f42'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:18.917029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1075/2142] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:29:18 2022] DELETE /identity/v3/users/79881618a76a40c6884aada4a6d10f42 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:29:18.921793 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c6c77-4d96-441b-99a5-a0bba53f5ac8 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:18.922291 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c6c77-4d96-441b-99a5-a0bba53f5ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:18.922380 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c6c77-4d96-441b-99a5-a0bba53f5ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:18.942954 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-212c6c77-4d96-441b-99a5-a0bba53f5ac8 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:18.944231 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-212c6c77-4d96-441b-99a5-a0bba53f5ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:18.994558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:29:19.011146 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1068/2143] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:29:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:29:19.021842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff50cd-bcaa-42c7-8285-0dbff8e70b21 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:19.023470 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a3ff50cd-bcaa-42c7-8285-0dbff8e70b21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:19.088576 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff50cd-bcaa-42c7-8285-0dbff8e70b21 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:19.092830 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86910e7b-37b8-47f2-8d20-f42208bc927a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:19.093446 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86910e7b-37b8-47f2-8d20-f42208bc927a service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:19.093725 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86910e7b-37b8-47f2-8d20-f42208bc927a service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:19.093996 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86910e7b-37b8-47f2-8d20-f42208bc927a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:19.108034 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86910e7b-37b8-47f2-8d20-f42208bc927a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:19.109521 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86910e7b-37b8-47f2-8d20-f42208bc927a service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:19.122196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1076/2144] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:29:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:29:19.635176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c6c77-4d96-441b-99a5-a0bba53f5ac8 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:19.637055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-212c6c77-4d96-441b-99a5-a0bba53f5ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:19.659857 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c6c77-4d96-441b-99a5-a0bba53f5ac8 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:19.671420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b84b08-0df2-4a72-85b3-0fefc2b65943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:19.672648 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b84b08-0df2-4a72-85b3-0fefc2b65943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:19.672940 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b84b08-0df2-4a72-85b3-0fefc2b65943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:19.673308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b84b08-0df2-4a72-85b3-0fefc2b65943 admin admin] PATH_INFO: `/v3/projects/78f8cb4784554c44aa3a89a9014c7a6d` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:19.674841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b84b08-0df2-4a72-85b3-0fefc2b65943 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78f8cb4784554c44aa3a89a9014c7a6d)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:19.677361 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b84b08-0df2-4a72-85b3-0fefc2b65943 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:19.755042 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-57b84b08-0df2-4a72-85b3-0fefc2b65943 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78f8cb4784554c44aa3a89a9014c7a6d', 'request_id': 'req-57b84b08-0df2-4a72-85b3-0fefc2b65943'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:19.762211 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-57b84b08-0df2-4a72-85b3-0fefc2b65943 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78f8cb4784554c44aa3a89a9014c7a6d', 'request_id': 'req-57b84b08-0df2-4a72-85b3-0fefc2b65943'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:19.771408 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-57b84b08-0df2-4a72-85b3-0fefc2b65943 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78f8cb4784554c44aa3a89a9014c7a6d', 'request_id': 'req-57b84b08-0df2-4a72-85b3-0fefc2b65943'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:19.783588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-57b84b08-0df2-4a72-85b3-0fefc2b65943 admin admin] The token cache is being invalidate because project 78f8cb4784554c44aa3a89a9014c7a6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:29:19.784006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-57b84b08-0df2-4a72-85b3-0fefc2b65943 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:19.785015 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-57b84b08-0df2-4a72-85b3-0fefc2b65943 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:19.786665 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1069/2145] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:29:19 2022] DELETE /identity/v3/projects/78f8cb4784554c44aa3a89a9014c7a6d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:29:19.790966 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86910e7b-37b8-47f2-8d20-f42208bc927a service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:19.803111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86910e7b-37b8-47f2-8d20-f42208bc927a service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:19.870538 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e98f57b-003e-45a5-9716-95bc7d12b2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:19.871276 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98f57b-003e-45a5-9716-95bc7d12b2aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:19.871557 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98f57b-003e-45a5-9716-95bc7d12b2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:19.871831 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98f57b-003e-45a5-9716-95bc7d12b2aa admin admin] PATH_INFO: `/v3/users/8e0c161c55cd4335844aa9d4c32ff62a` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:19.872996 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e98f57b-003e-45a5-9716-95bc7d12b2aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e0c161c55cd4335844aa9d4c32ff62a)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:19.874949 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e98f57b-003e-45a5-9716-95bc7d12b2aa admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:19.927244 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-2e98f57b-003e-45a5-9716-95bc7d12b2aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e0c161c55cd4335844aa9d4c32ff62a'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:19.927889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-2e98f57b-003e-45a5-9716-95bc7d12b2aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e0c161c55cd4335844aa9d4c32ff62a'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:19.935123 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-2e98f57b-003e-45a5-9716-95bc7d12b2aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e0c161c55cd4335844aa9d4c32ff62a'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:19.947370 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-2e98f57b-003e-45a5-9716-95bc7d12b2aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e0c161c55cd4335844aa9d4c32ff62a'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:19.948570 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:29:19.956959 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-2e98f57b-003e-45a5-9716-95bc7d12b2aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e0c161c55cd4335844aa9d4c32ff62a'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:19.963466 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1077/2146] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:29:19 2022] DELETE /identity/v3/users/8e0c161c55cd4335844aa9d4c32ff62a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:29:20.266009 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b84b08-0df2-4a72-85b3-0fefc2b65943 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:20.267648 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-57b84b08-0df2-4a72-85b3-0fefc2b65943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:20.281370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b84b08-0df2-4a72-85b3-0fefc2b65943 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:20.342571 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a340f6-e66f-40bd-9b59-f4ffb5173fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:20.343219 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a340f6-e66f-40bd-9b59-f4ffb5173fb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:20.343494 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a340f6-e66f-40bd-9b59-f4ffb5173fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:20.343762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a340f6-e66f-40bd-9b59-f4ffb5173fb2 admin admin] PATH_INFO: `/v3/projects/af45a7c156b844eeb2c4ef506eb1a86e` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:20.344903 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a340f6-e66f-40bd-9b59-f4ffb5173fb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af45a7c156b844eeb2c4ef506eb1a86e)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:20.346861 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a340f6-e66f-40bd-9b59-f4ffb5173fb2 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:20.398124 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-45a340f6-e66f-40bd-9b59-f4ffb5173fb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af45a7c156b844eeb2c4ef506eb1a86e', 'request_id': 'req-45a340f6-e66f-40bd-9b59-f4ffb5173fb2'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:20.404903 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-45a340f6-e66f-40bd-9b59-f4ffb5173fb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af45a7c156b844eeb2c4ef506eb1a86e', 'request_id': 'req-45a340f6-e66f-40bd-9b59-f4ffb5173fb2'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:20.415661 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-45a340f6-e66f-40bd-9b59-f4ffb5173fb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af45a7c156b844eeb2c4ef506eb1a86e', 'request_id': 'req-45a340f6-e66f-40bd-9b59-f4ffb5173fb2'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:20.426455 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-45a340f6-e66f-40bd-9b59-f4ffb5173fb2 admin admin] The token cache is being invalidate because project af45a7c156b844eeb2c4ef506eb1a86e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:29:20.426746 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-45a340f6-e66f-40bd-9b59-f4ffb5173fb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:20.427389 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-45a340f6-e66f-40bd-9b59-f4ffb5173fb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:20.428933 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1070/2147] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:29:20 2022] DELETE /identity/v3/projects/af45a7c156b844eeb2c4ef506eb1a86e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:29:20.434110 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f122d5d7-6f29-45ea-b474-c287e35d7a71 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:20.434574 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f122d5d7-6f29-45ea-b474-c287e35d7a71 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:20.434934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f122d5d7-6f29-45ea-b474-c287e35d7a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:20.456712 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-f122d5d7-6f29-45ea-b474-c287e35d7a71 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:20.462567 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-f122d5d7-6f29-45ea-b474-c287e35d7a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:20.527728 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1078/2148] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:29:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:29:20.537045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a7ec7-65e7-4740-999f-4224db76b5d6 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:20.537430 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a7ec7-65e7-4740-999f-4224db76b5d6 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:20.537782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a7ec7-65e7-4740-999f-4224db76b5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:20.546875 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:29:20.559819 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-bf4a7ec7-65e7-4740-999f-4224db76b5d6 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:20.561805 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-bf4a7ec7-65e7-4740-999f-4224db76b5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:20.570707 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1071/2149] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:29:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:29:20.575421 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f122d5d7-6f29-45ea-b474-c287e35d7a71 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:20.587577 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f122d5d7-6f29-45ea-b474-c287e35d7a71 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:20.592156 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d50b8-cefa-4666-9e16-6b98264554e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:20.592756 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d50b8-cefa-4666-9e16-6b98264554e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:20.593027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d50b8-cefa-4666-9e16-6b98264554e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:20.593290 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d50b8-cefa-4666-9e16-6b98264554e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:20.593931 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d50b8-cefa-4666-9e16-6b98264554e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:29:20.594231 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d50b8-cefa-4666-9e16-6b98264554e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:20.596123 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d50b8-cefa-4666-9e16-6b98264554e7 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:20.608386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1079/2150] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:29:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:29:20.613154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a7ec7-65e7-4740-999f-4224db76b5d6 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:20.622612 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a7ec7-65e7-4740-999f-4224db76b5d6 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:20.627456 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd56b4-5c53-4f66-b9de-f75f6b009295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:20.628059 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd56b4-5c53-4f66-b9de-f75f6b009295 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:20.628332 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd56b4-5c53-4f66-b9de-f75f6b009295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:20.628586 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd56b4-5c53-4f66-b9de-f75f6b009295 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:20.629237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fd56b4-5c53-4f66-b9de-f75f6b009295 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:20.631114 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fd56b4-5c53-4f66-b9de-f75f6b009295 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:20.660950 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1072/2151] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:29:20 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:29:20.665139 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d50b8-cefa-4666-9e16-6b98264554e7 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:20.675517 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d50b8-cefa-4666-9e16-6b98264554e7 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:20.734374 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a287fd-d63c-4d02-a7e8-ac062f7dd757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:20.735016 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a287fd-d63c-4d02-a7e8-ac062f7dd757 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:20.735295 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a287fd-d63c-4d02-a7e8-ac062f7dd757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:20.735555 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a287fd-d63c-4d02-a7e8-ac062f7dd757 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:20.736284 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a287fd-d63c-4d02-a7e8-ac062f7dd757 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:20.738210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a287fd-d63c-4d02-a7e8-ac062f7dd757 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:20.760272 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1080/2152] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:29:20 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:29:20.764424 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd56b4-5c53-4f66-b9de-f75f6b009295 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:20.774813 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd56b4-5c53-4f66-b9de-f75f6b009295 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:20.781482 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbec942-5f4e-48d7-9677-6dcc371a77d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:20.782416 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbec942-5f4e-48d7-9677-6dcc371a77d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:20.782742 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbec942-5f4e-48d7-9677-6dcc371a77d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:20.783101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbec942-5f4e-48d7-9677-6dcc371a77d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:20.783749 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbec942-5f4e-48d7-9677-6dcc371a77d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:20.785439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbec942-5f4e-48d7-9677-6dcc371a77d7 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:20.795884 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1073/2153] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:29:20.800126 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a287fd-d63c-4d02-a7e8-ac062f7dd757 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:20.810231 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a287fd-d63c-4d02-a7e8-ac062f7dd757 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:20.816063 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346cbf3-68a2-4d56-8238-c430633e9e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:20.816666 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346cbf3-68a2-4d56-8238-c430633e9e0e admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:20.816946 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346cbf3-68a2-4d56-8238-c430633e9e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:20.817251 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346cbf3-68a2-4d56-8238-c430633e9e0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:20.817917 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7346cbf3-68a2-4d56-8238-c430633e9e0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:20.819552 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7346cbf3-68a2-4d56-8238-c430633e9e0e admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:20.830722 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1081/2154] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:29:20.837021 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbec942-5f4e-48d7-9677-6dcc371a77d7 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:20.853698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbec942-5f4e-48d7-9677-6dcc371a77d7 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:20.861653 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f5b32-b6bc-4c4e-a712-26e6f1cd0553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:20.862310 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f5b32-b6bc-4c4e-a712-26e6f1cd0553 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:20.862586 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f5b32-b6bc-4c4e-a712-26e6f1cd0553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:20.862867 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f5b32-b6bc-4c4e-a712-26e6f1cd0553 admin admin] PATH_INFO: `/v3/projects/92a3c6095792435c8d8b69fd8cab9bf1/users/bca35fd757434a9e94219a8672cfb3ea/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:20.878897 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f5b32-b6bc-4c4e-a712-26e6f1cd0553 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92a3c6095792435c8d8b69fd8cab9bf1, user_id=bca35fd757434a9e94219a8672cfb3ea, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:20.881583 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f5b32-b6bc-4c4e-a712-26e6f1cd0553 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:20.897587 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1074/2155] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:29:20 2022] PUT /identity/v3/projects/92a3c6095792435c8d8b69fd8cab9bf1/users/bca35fd757434a9e94219a8672cfb3ea/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:29:20.907806 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c5202-10a1-4ea7-af10-7c92354de0d5 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:20.908191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c5202-10a1-4ea7-af10-7c92354de0d5 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:20.908446 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c5202-10a1-4ea7-af10-7c92354de0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:20.941494 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-fa1c5202-10a1-4ea7-af10-7c92354de0d5 None None] MFA Rules not processed for user `bca35fd757434a9e94219a8672cfb3ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:20.942839 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-fa1c5202-10a1-4ea7-af10-7c92354de0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:20.994041 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:29:21.015279 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1082/2156] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:29:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:29:21.026811 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f5b32-b6bc-4c4e-a712-26e6f1cd0553 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:21.028695 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-c76f5b32-b6bc-4c4e-a712-26e6f1cd0553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:21.088041 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f5b32-b6bc-4c4e-a712-26e6f1cd0553 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:21.091626 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c21562-c05c-447a-b75a-ce58f2c01871 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:21.092301 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c21562-c05c-447a-b75a-ce58f2c01871 service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:21.092580 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c21562-c05c-447a-b75a-ce58f2c01871 service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:21.092854 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c21562-c05c-447a-b75a-ce58f2c01871 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:21.101363 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c21562-c05c-447a-b75a-ce58f2c01871 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:21.102800 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c21562-c05c-447a-b75a-ce58f2c01871 service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:21.111508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1075/2157] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:29:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:29:21.132108 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c5202-10a1-4ea7-af10-7c92354de0d5 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:21.199625 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c5202-10a1-4ea7-af10-7c92354de0d5 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:21.204614 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f59eb-7d4e-4bb2-9c3b-b451fc57c386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:21.205489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f59eb-7d4e-4bb2-9c3b-b451fc57c386 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:21.205761 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f59eb-7d4e-4bb2-9c3b-b451fc57c386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:21.206036 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f59eb-7d4e-4bb2-9c3b-b451fc57c386 admin admin] PATH_INFO: `/v3/users/bca35fd757434a9e94219a8672cfb3ea` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:21.207250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f59eb-7d4e-4bb2-9c3b-b451fc57c386 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bca35fd757434a9e94219a8672cfb3ea)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:21.209239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f59eb-7d4e-4bb2-9c3b-b451fc57c386 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:21.259826 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c93f59eb-7d4e-4bb2-9c3b-b451fc57c386 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bca35fd757434a9e94219a8672cfb3ea'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:21.260749 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c93f59eb-7d4e-4bb2-9c3b-b451fc57c386 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bca35fd757434a9e94219a8672cfb3ea'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:21.270471 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c93f59eb-7d4e-4bb2-9c3b-b451fc57c386 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bca35fd757434a9e94219a8672cfb3ea'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:21.285097 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c93f59eb-7d4e-4bb2-9c3b-b451fc57c386 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bca35fd757434a9e94219a8672cfb3ea'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:21.292499 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c93f59eb-7d4e-4bb2-9c3b-b451fc57c386 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bca35fd757434a9e94219a8672cfb3ea'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:21.298890 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1083/2158] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:29:21 2022] DELETE /identity/v3/users/bca35fd757434a9e94219a8672cfb3ea => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:29:21.303501 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d213f-31c9-4ed3-b133-a0eadad5dfb4 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:21.304016 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d213f-31c9-4ed3-b133-a0eadad5dfb4 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:21.304125 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d213f-31c9-4ed3-b133-a0eadad5dfb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:21.323153 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-3e7d213f-31c9-4ed3-b133-a0eadad5dfb4 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:21.324336 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-3e7d213f-31c9-4ed3-b133-a0eadad5dfb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:21.384890 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1076/2159] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:29:21.399082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f59eb-7d4e-4bb2-9c3b-b451fc57c386 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:21.401465 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-c93f59eb-7d4e-4bb2-9c3b-b451fc57c386 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:21.465282 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f59eb-7d4e-4bb2-9c3b-b451fc57c386 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:21.469659 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc6b99-3fb7-4224-857f-8f76126cf771 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:21.470274 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc6b99-3fb7-4224-857f-8f76126cf771 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:21.470522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc6b99-3fb7-4224-857f-8f76126cf771 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:21.471257 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc6b99-3fb7-4224-857f-8f76126cf771 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:21.481761 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc6b99-3fb7-4224-857f-8f76126cf771 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:21.483203 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc6b99-3fb7-4224-857f-8f76126cf771 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:21.493761 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1084/2160] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:29:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:29:21.776562 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d213f-31c9-4ed3-b133-a0eadad5dfb4 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:21.778901 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-3e7d213f-31c9-4ed3-b133-a0eadad5dfb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:21.811899 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d213f-31c9-4ed3-b133-a0eadad5dfb4 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:21.817311 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6c79f-3d4f-4223-b744-04174b5a8de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:21.817953 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6c79f-3d4f-4223-b744-04174b5a8de5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:21.818230 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6c79f-3d4f-4223-b744-04174b5a8de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:21.818496 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6c79f-3d4f-4223-b744-04174b5a8de5 admin admin] PATH_INFO: `/v3/projects/92a3c6095792435c8d8b69fd8cab9bf1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:21.819615 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6c79f-3d4f-4223-b744-04174b5a8de5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92a3c6095792435c8d8b69fd8cab9bf1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:21.821567 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6c79f-3d4f-4223-b744-04174b5a8de5 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:21.865539 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:29:21.874044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-4bb6c79f-3d4f-4223-b744-04174b5a8de5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92a3c6095792435c8d8b69fd8cab9bf1', 'request_id': 'req-4bb6c79f-3d4f-4223-b744-04174b5a8de5'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:21.880789 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-4bb6c79f-3d4f-4223-b744-04174b5a8de5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92a3c6095792435c8d8b69fd8cab9bf1', 'request_id': 'req-4bb6c79f-3d4f-4223-b744-04174b5a8de5'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:21.889232 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-4bb6c79f-3d4f-4223-b744-04174b5a8de5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92a3c6095792435c8d8b69fd8cab9bf1', 'request_id': 'req-4bb6c79f-3d4f-4223-b744-04174b5a8de5'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:21.900050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-4bb6c79f-3d4f-4223-b744-04174b5a8de5 admin admin] The token cache is being invalidate because project 92a3c6095792435c8d8b69fd8cab9bf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:29:21.900342 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-4bb6c79f-3d4f-4223-b744-04174b5a8de5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:21.901004 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-4bb6c79f-3d4f-4223-b744-04174b5a8de5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:21.902623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1077/2161] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:29:21 2022] DELETE /identity/v3/projects/92a3c6095792435c8d8b69fd8cab9bf1 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:29:21.907540 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9adf01-9829-443c-938c-3012054d1c5e None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:21.908040 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9adf01-9829-443c-938c-3012054d1c5e None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:21.908183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9adf01-9829-443c-938c-3012054d1c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:21.922512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:29:21.932164 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-2d9adf01-9829-443c-938c-3012054d1c5e None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:21.934508 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-2d9adf01-9829-443c-938c-3012054d1c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:22.002478 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1085/2162] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:29:22.011316 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e9755-fced-493d-a93f-13cf4eb81393 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:22.011601 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e9755-fced-493d-a93f-13cf4eb81393 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:22.011871 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e9755-fced-493d-a93f-13cf4eb81393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:22.033588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-9b1e9755-fced-493d-a93f-13cf4eb81393 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:22.034789 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-9b1e9755-fced-493d-a93f-13cf4eb81393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:22.044189 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1078/2163] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:29:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:29:22.048586 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9adf01-9829-443c-938c-3012054d1c5e None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:22.060648 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9adf01-9829-443c-938c-3012054d1c5e None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:22.066851 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f859b9-6ae7-488b-acc9-26be08a9ba5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:22.067458 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f859b9-6ae7-488b-acc9-26be08a9ba5b admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:22.067715 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f859b9-6ae7-488b-acc9-26be08a9ba5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:22.068012 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f859b9-6ae7-488b-acc9-26be08a9ba5b admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:22.068645 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f859b9-6ae7-488b-acc9-26be08a9ba5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:29:22.068943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f859b9-6ae7-488b-acc9-26be08a9ba5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:22.070554 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f859b9-6ae7-488b-acc9-26be08a9ba5b admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:22.082185 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1086/2164] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:29:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:29:22.086413 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e9755-fced-493d-a93f-13cf4eb81393 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:22.095816 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e9755-fced-493d-a93f-13cf4eb81393 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:22.101606 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe34fe-2869-4751-a711-764be3dd8371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:22.102210 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe34fe-2869-4751-a711-764be3dd8371 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:22.102483 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe34fe-2869-4751-a711-764be3dd8371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:22.102753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe34fe-2869-4751-a711-764be3dd8371 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:22.103360 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe34fe-2869-4751-a711-764be3dd8371 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:22.105378 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe34fe-2869-4751-a711-764be3dd8371 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:22.134519 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1079/2165] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:29:22 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:29:22.138747 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f859b9-6ae7-488b-acc9-26be08a9ba5b admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:22.149155 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f859b9-6ae7-488b-acc9-26be08a9ba5b admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:22.208138 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c15b4-4b79-477e-baa7-b8047a3e0cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:22.208737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c15b4-4b79-477e-baa7-b8047a3e0cb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:22.209011 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c15b4-4b79-477e-baa7-b8047a3e0cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:22.209277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c15b4-4b79-477e-baa7-b8047a3e0cb2 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:22.209898 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c15b4-4b79-477e-baa7-b8047a3e0cb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:22.211806 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c15b4-4b79-477e-baa7-b8047a3e0cb2 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:22.234935 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1087/2166] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:29:22 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:29:22.239353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe34fe-2869-4751-a711-764be3dd8371 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:22.249566 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe34fe-2869-4751-a711-764be3dd8371 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:22.254308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d6dc7-25a0-416a-9873-69d60b506a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:22.254923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d6dc7-25a0-416a-9873-69d60b506a80 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:22.255200 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d6dc7-25a0-416a-9873-69d60b506a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:22.255482 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d6dc7-25a0-416a-9873-69d60b506a80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:22.256142 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d6dc7-25a0-416a-9873-69d60b506a80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:22.257731 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d6dc7-25a0-416a-9873-69d60b506a80 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:22.268255 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1080/2167] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:29:22.272435 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c15b4-4b79-477e-baa7-b8047a3e0cb2 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:22.282277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c15b4-4b79-477e-baa7-b8047a3e0cb2 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:22.287101 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea890e8-0fed-48cf-99ee-d00ba4bbdd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:22.287691 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea890e8-0fed-48cf-99ee-d00ba4bbdd07 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:22.288003 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea890e8-0fed-48cf-99ee-d00ba4bbdd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:22.288276 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea890e8-0fed-48cf-99ee-d00ba4bbdd07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:22.288920 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea890e8-0fed-48cf-99ee-d00ba4bbdd07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:22.290513 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea890e8-0fed-48cf-99ee-d00ba4bbdd07 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:22.301505 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1088/2168] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:29:22.305733 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d6dc7-25a0-416a-9873-69d60b506a80 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:22.316649 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d6dc7-25a0-416a-9873-69d60b506a80 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:22.321811 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a069e-d62c-4252-a793-e76130808dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:22.322493 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a069e-d62c-4252-a793-e76130808dd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:22.322768 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a069e-d62c-4252-a793-e76130808dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:22.323035 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a069e-d62c-4252-a793-e76130808dd6 admin admin] PATH_INFO: `/v3/projects/2566152ea8d24db8b4e8d1294449445f/users/bf96fc2f8b8b4debb6a5488aa72448f2/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:22.338274 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2a069e-d62c-4252-a793-e76130808dd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2566152ea8d24db8b4e8d1294449445f, user_id=bf96fc2f8b8b4debb6a5488aa72448f2, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:22.340991 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2a069e-d62c-4252-a793-e76130808dd6 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:22.354546 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1081/2169] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:29:22 2022] PUT /identity/v3/projects/2566152ea8d24db8b4e8d1294449445f/users/bf96fc2f8b8b4debb6a5488aa72448f2/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:29:22.364887 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc448fd8-df9c-4692-9726-a45eefe6f541 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:22.365180 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc448fd8-df9c-4692-9726-a45eefe6f541 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:22.365454 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc448fd8-df9c-4692-9726-a45eefe6f541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:22.394842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-fc448fd8-df9c-4692-9726-a45eefe6f541 None None] MFA Rules not processed for user `bf96fc2f8b8b4debb6a5488aa72448f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:22.396096 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-fc448fd8-df9c-4692-9726-a45eefe6f541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:22.468576 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1089/2170] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:29:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:29:22.480199 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a069e-d62c-4252-a793-e76130808dd6 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:22.482149 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-ff2a069e-d62c-4252-a793-e76130808dd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:22.545405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a069e-d62c-4252-a793-e76130808dd6 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:22.549540 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a2e62-90e9-48ad-aa76-5c4dbe63a683 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:22.550162 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a2e62-90e9-48ad-aa76-5c4dbe63a683 service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:22.550439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a2e62-90e9-48ad-aa76-5c4dbe63a683 service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:22.550714 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a2e62-90e9-48ad-aa76-5c4dbe63a683 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:22.559768 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88a2e62-90e9-48ad-aa76-5c4dbe63a683 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:22.561394 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88a2e62-90e9-48ad-aa76-5c4dbe63a683 service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:22.570799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1082/2171] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:29:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:29:22.592011 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc448fd8-df9c-4692-9726-a45eefe6f541 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:22.596416 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:29:22.655800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc448fd8-df9c-4692-9726-a45eefe6f541 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:22.660533 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a7b80-4a2a-42b6-b98c-a2a467de5f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:22.661197 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a7b80-4a2a-42b6-b98c-a2a467de5f8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:22.661468 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a7b80-4a2a-42b6-b98c-a2a467de5f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:22.661735 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a7b80-4a2a-42b6-b98c-a2a467de5f8f admin admin] PATH_INFO: `/v3/users/bf96fc2f8b8b4debb6a5488aa72448f2` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:22.662955 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a7b80-4a2a-42b6-b98c-a2a467de5f8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf96fc2f8b8b4debb6a5488aa72448f2)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:22.665089 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a7b80-4a2a-42b6-b98c-a2a467de5f8f admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:22.725947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-843a7b80-4a2a-42b6-b98c-a2a467de5f8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf96fc2f8b8b4debb6a5488aa72448f2'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:22.726628 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-843a7b80-4a2a-42b6-b98c-a2a467de5f8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf96fc2f8b8b4debb6a5488aa72448f2'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:22.734483 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-843a7b80-4a2a-42b6-b98c-a2a467de5f8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf96fc2f8b8b4debb6a5488aa72448f2'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:22.750055 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-843a7b80-4a2a-42b6-b98c-a2a467de5f8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf96fc2f8b8b4debb6a5488aa72448f2'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:22.757636 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-843a7b80-4a2a-42b6-b98c-a2a467de5f8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf96fc2f8b8b4debb6a5488aa72448f2'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:22.763931 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1090/2172] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:29:22 2022] DELETE /identity/v3/users/bf96fc2f8b8b4debb6a5488aa72448f2 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:29:22.768725 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39f933-0590-476f-918f-ae6373ff95be None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:22.769015 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39f933-0590-476f-918f-ae6373ff95be None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:22.769310 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39f933-0590-476f-918f-ae6373ff95be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:22.788819 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-7f39f933-0590-476f-918f-ae6373ff95be None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:22.790029 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-7f39f933-0590-476f-918f-ae6373ff95be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:22.857172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1083/2173] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:29:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:29:22.868268 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a7b80-4a2a-42b6-b98c-a2a467de5f8f admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:22.869884 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-843a7b80-4a2a-42b6-b98c-a2a467de5f8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:22.933202 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a7b80-4a2a-42b6-b98c-a2a467de5f8f admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:22.937430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855655b6-a7e3-4f1a-9b6e-9b945eb85357 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:22.938133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855655b6-a7e3-4f1a-9b6e-9b945eb85357 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:22.938435 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855655b6-a7e3-4f1a-9b6e-9b945eb85357 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:22.938705 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855655b6-a7e3-4f1a-9b6e-9b945eb85357 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:22.949906 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855655b6-a7e3-4f1a-9b6e-9b945eb85357 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:22.951429 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855655b6-a7e3-4f1a-9b6e-9b945eb85357 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:22.964061 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1091/2174] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:29:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:29:23.306105 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39f933-0590-476f-918f-ae6373ff95be None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:23.307610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-7f39f933-0590-476f-918f-ae6373ff95be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:23.320533 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39f933-0590-476f-918f-ae6373ff95be None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:23.331478 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0216377-08a9-4940-bd3f-7103f6daf8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:23.332236 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0216377-08a9-4940-bd3f-7103f6daf8a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:23.332548 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0216377-08a9-4940-bd3f-7103f6daf8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:23.332822 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0216377-08a9-4940-bd3f-7103f6daf8a3 admin admin] PATH_INFO: `/v3/projects/2566152ea8d24db8b4e8d1294449445f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:23.334048 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0216377-08a9-4940-bd3f-7103f6daf8a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2566152ea8d24db8b4e8d1294449445f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:23.336039 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0216377-08a9-4940-bd3f-7103f6daf8a3 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:23.337918 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:29:23.390425 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f0216377-08a9-4940-bd3f-7103f6daf8a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2566152ea8d24db8b4e8d1294449445f', 'request_id': 'req-f0216377-08a9-4940-bd3f-7103f6daf8a3'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:23.398363 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f0216377-08a9-4940-bd3f-7103f6daf8a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2566152ea8d24db8b4e8d1294449445f', 'request_id': 'req-f0216377-08a9-4940-bd3f-7103f6daf8a3'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:23.406397 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f0216377-08a9-4940-bd3f-7103f6daf8a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2566152ea8d24db8b4e8d1294449445f', 'request_id': 'req-f0216377-08a9-4940-bd3f-7103f6daf8a3'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:23.417582 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f0216377-08a9-4940-bd3f-7103f6daf8a3 admin admin] The token cache is being invalidate because project 2566152ea8d24db8b4e8d1294449445f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:29:23.417874 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f0216377-08a9-4940-bd3f-7103f6daf8a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:23.418760 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f0216377-08a9-4940-bd3f-7103f6daf8a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:23.420501 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1084/2175] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:29:23 2022] DELETE /identity/v3/projects/2566152ea8d24db8b4e8d1294449445f => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:29:23.425851 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2ec79-5715-4542-8ff4-140dc2adc602 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:23.426142 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2ec79-5715-4542-8ff4-140dc2adc602 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:23.426419 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2ec79-5715-4542-8ff4-140dc2adc602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:23.447971 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-e9f2ec79-5715-4542-8ff4-140dc2adc602 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:23.449156 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-e9f2ec79-5715-4542-8ff4-140dc2adc602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:23.470109 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:29:23.513435 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1092/2176] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:29:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:29:23.521974 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7b8fb-b2d4-4a40-b703-588be4aad2b2 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:23.522301 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7b8fb-b2d4-4a40-b703-588be4aad2b2 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:23.522581 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7b8fb-b2d4-4a40-b703-588be4aad2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:23.543149 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-07e7b8fb-b2d4-4a40-b703-588be4aad2b2 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:23.544797 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-07e7b8fb-b2d4-4a40-b703-588be4aad2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:23.553248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1085/2177] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:29:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:29:23.557469 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2ec79-5715-4542-8ff4-140dc2adc602 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:23.568143 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2ec79-5715-4542-8ff4-140dc2adc602 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:23.573101 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ab3be-99cb-40f3-a277-9b60adc3671a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:23.573727 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ab3be-99cb-40f3-a277-9b60adc3671a admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:23.574027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ab3be-99cb-40f3-a277-9b60adc3671a admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:23.574295 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ab3be-99cb-40f3-a277-9b60adc3671a admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:23.574975 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ab3be-99cb-40f3-a277-9b60adc3671a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:29:23.575280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ab3be-99cb-40f3-a277-9b60adc3671a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:23.576959 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ab3be-99cb-40f3-a277-9b60adc3671a admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:23.593497 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1093/2178] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:29:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:29:23.595125 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7b8fb-b2d4-4a40-b703-588be4aad2b2 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:23.605528 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7b8fb-b2d4-4a40-b703-588be4aad2b2 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:23.610327 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f18297-8a6c-44e9-bf4a-03c338da19f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:23.610927 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f18297-8a6c-44e9-bf4a-03c338da19f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:23.611226 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f18297-8a6c-44e9-bf4a-03c338da19f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:23.611496 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f18297-8a6c-44e9-bf4a-03c338da19f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:23.612788 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f18297-8a6c-44e9-bf4a-03c338da19f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:23.614701 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f18297-8a6c-44e9-bf4a-03c338da19f6 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:23.644290 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1086/2179] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:29:23 2022] POST /identity/v3/projects => generated 361 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:29:23.648920 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ab3be-99cb-40f3-a277-9b60adc3671a admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:23.660765 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ab3be-99cb-40f3-a277-9b60adc3671a admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:23.723764 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a369f0-91d0-437f-966f-21508ec651f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:23.724337 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a369f0-91d0-437f-966f-21508ec651f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:23.724639 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a369f0-91d0-437f-966f-21508ec651f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:23.724870 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a369f0-91d0-437f-966f-21508ec651f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:23.725496 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a369f0-91d0-437f-966f-21508ec651f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:23.727430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a369f0-91d0-437f-966f-21508ec651f5 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:23.752644 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1094/2180] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:29:23 2022] POST /identity/v3/users => generated 281 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:29:23.756739 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f18297-8a6c-44e9-bf4a-03c338da19f6 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:23.767432 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f18297-8a6c-44e9-bf4a-03c338da19f6 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:23.772672 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba89b759-fade-47bd-ab32-0ab0ef4889bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:23.773276 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89b759-fade-47bd-ab32-0ab0ef4889bd admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:23.773607 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89b759-fade-47bd-ab32-0ab0ef4889bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:23.773881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89b759-fade-47bd-ab32-0ab0ef4889bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:23.774527 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba89b759-fade-47bd-ab32-0ab0ef4889bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:23.776160 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba89b759-fade-47bd-ab32-0ab0ef4889bd admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:23.786457 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1087/2181] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:29:23.790442 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a369f0-91d0-437f-966f-21508ec651f5 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:23.800912 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a369f0-91d0-437f-966f-21508ec651f5 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:23.806330 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8f618-608d-422c-a268-793f714c0602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:23.806964 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8f618-608d-422c-a268-793f714c0602 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:23.807238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8f618-608d-422c-a268-793f714c0602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:23.807506 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8f618-608d-422c-a268-793f714c0602 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:23.808221 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8f618-608d-422c-a268-793f714c0602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:23.809869 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8f618-608d-422c-a268-793f714c0602 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:23.821537 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1095/2182] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:29:23.826042 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba89b759-fade-47bd-ab32-0ab0ef4889bd admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:23.837044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba89b759-fade-47bd-ab32-0ab0ef4889bd admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:23.841908 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e45be-b18d-4364-a24f-4e009f08fed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:23.842554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e45be-b18d-4364-a24f-4e009f08fed5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:23.842830 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e45be-b18d-4364-a24f-4e009f08fed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:23.843104 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e45be-b18d-4364-a24f-4e009f08fed5 admin admin] PATH_INFO: `/v3/projects/1279a8705bf445f68b6b41a695b6cf8c/users/85f5349396494ae8855a5d5a4d65254b/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:23.859097 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9e45be-b18d-4364-a24f-4e009f08fed5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1279a8705bf445f68b6b41a695b6cf8c, user_id=85f5349396494ae8855a5d5a4d65254b, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:23.862367 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9e45be-b18d-4364-a24f-4e009f08fed5 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:23.879059 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1088/2183] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:29:23 2022] PUT /identity/v3/projects/1279a8705bf445f68b6b41a695b6cf8c/users/85f5349396494ae8855a5d5a4d65254b/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:29:23.890530 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656ca59-b989-4c42-be1d-daee77c9c6cc None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:23.890814 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656ca59-b989-4c42-be1d-daee77c9c6cc None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:23.891093 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656ca59-b989-4c42-be1d-daee77c9c6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:23.931173 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-b656ca59-b989-4c42-be1d-daee77c9c6cc None None] MFA Rules not processed for user `85f5349396494ae8855a5d5a4d65254b`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:23.932439 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-b656ca59-b989-4c42-be1d-daee77c9c6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:24.005253 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1096/2184] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:29:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:29:24.039854 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e45be-b18d-4364-a24f-4e009f08fed5 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:24.041960 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-7d9e45be-b18d-4364-a24f-4e009f08fed5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:24.110863 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e45be-b18d-4364-a24f-4e009f08fed5 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:24.115595 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1031ec24-7c33-408a-a771-a125bd6478d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:24.116254 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031ec24-7c33-408a-a771-a125bd6478d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:24.116556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031ec24-7c33-408a-a771-a125bd6478d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:24.116862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031ec24-7c33-408a-a771-a125bd6478d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:24.126306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1031ec24-7c33-408a-a771-a125bd6478d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:24.127790 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1031ec24-7c33-408a-a771-a125bd6478d6 service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:24.138767 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1089/2185] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:29:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:29:24.155327 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656ca59-b989-4c42-be1d-daee77c9c6cc None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:24.180880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:29:24.228819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656ca59-b989-4c42-be1d-daee77c9c6cc None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:24.233741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f655c9-3ccc-486e-b438-ef77ca483c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:24.234400 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f655c9-3ccc-486e-b438-ef77ca483c51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:24.234681 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f655c9-3ccc-486e-b438-ef77ca483c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:24.234950 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f655c9-3ccc-486e-b438-ef77ca483c51 admin admin] PATH_INFO: `/v3/users/85f5349396494ae8855a5d5a4d65254b` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:24.236153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f655c9-3ccc-486e-b438-ef77ca483c51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85f5349396494ae8855a5d5a4d65254b)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:24.238235 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f655c9-3ccc-486e-b438-ef77ca483c51 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:24.291330 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-32f655c9-3ccc-486e-b438-ef77ca483c51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85f5349396494ae8855a5d5a4d65254b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:24.291943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-32f655c9-3ccc-486e-b438-ef77ca483c51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85f5349396494ae8855a5d5a4d65254b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:24.301070 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-32f655c9-3ccc-486e-b438-ef77ca483c51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85f5349396494ae8855a5d5a4d65254b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:24.313592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-32f655c9-3ccc-486e-b438-ef77ca483c51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85f5349396494ae8855a5d5a4d65254b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:24.320782 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-32f655c9-3ccc-486e-b438-ef77ca483c51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85f5349396494ae8855a5d5a4d65254b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:24.327075 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1097/2186] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:29:24 2022] DELETE /identity/v3/users/85f5349396494ae8855a5d5a4d65254b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:29:24.331556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab73542-43cc-41e3-979c-1cc4b6e170b2 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:24.331966 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab73542-43cc-41e3-979c-1cc4b6e170b2 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:24.332220 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab73542-43cc-41e3-979c-1cc4b6e170b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:24.354409 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-1ab73542-43cc-41e3-979c-1cc4b6e170b2 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:24.355646 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-1ab73542-43cc-41e3-979c-1cc4b6e170b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:24.380725 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:29:24.421574 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1090/2187] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:29:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:29:24.431216 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f655c9-3ccc-486e-b438-ef77ca483c51 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:24.432629 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-32f655c9-3ccc-486e-b438-ef77ca483c51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:24.495373 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f655c9-3ccc-486e-b438-ef77ca483c51 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:24.499270 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ce991-3e59-48c3-ba6b-1897224b3d5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:24.499881 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ce991-3e59-48c3-ba6b-1897224b3d5f service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:24.500177 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ce991-3e59-48c3-ba6b-1897224b3d5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:24.500447 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ce991-3e59-48c3-ba6b-1897224b3d5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:24.510484 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779ce991-3e59-48c3-ba6b-1897224b3d5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:24.511930 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779ce991-3e59-48c3-ba6b-1897224b3d5f service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:24.522578 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1098/2188] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:29:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:29:24.818141 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab73542-43cc-41e3-979c-1cc4b6e170b2 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:24.820892 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-1ab73542-43cc-41e3-979c-1cc4b6e170b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:24.835276 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab73542-43cc-41e3-979c-1cc4b6e170b2 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:24.840834 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be1c4ad-5a51-4613-86d3-59abb7338cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:24.841855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1c4ad-5a51-4613-86d3-59abb7338cbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:24.841855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1c4ad-5a51-4613-86d3-59abb7338cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:24.842042 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1c4ad-5a51-4613-86d3-59abb7338cbb admin admin] PATH_INFO: `/v3/projects/1279a8705bf445f68b6b41a695b6cf8c` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:24.843309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be1c4ad-5a51-4613-86d3-59abb7338cbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=1279a8705bf445f68b6b41a695b6cf8c)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:24.845348 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be1c4ad-5a51-4613-86d3-59abb7338cbb admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:24.896461 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-1be1c4ad-5a51-4613-86d3-59abb7338cbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1279a8705bf445f68b6b41a695b6cf8c', 'request_id': 'req-1be1c4ad-5a51-4613-86d3-59abb7338cbb'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:24.902677 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-1be1c4ad-5a51-4613-86d3-59abb7338cbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1279a8705bf445f68b6b41a695b6cf8c', 'request_id': 'req-1be1c4ad-5a51-4613-86d3-59abb7338cbb'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:24.912020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-1be1c4ad-5a51-4613-86d3-59abb7338cbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1279a8705bf445f68b6b41a695b6cf8c', 'request_id': 'req-1be1c4ad-5a51-4613-86d3-59abb7338cbb'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:24.928316 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-1be1c4ad-5a51-4613-86d3-59abb7338cbb admin admin] The token cache is being invalidate because project 1279a8705bf445f68b6b41a695b6cf8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:29:24.928677 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-1be1c4ad-5a51-4613-86d3-59abb7338cbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:24.929602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-1be1c4ad-5a51-4613-86d3-59abb7338cbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:24.931939 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1091/2189] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:29:24 2022] DELETE /identity/v3/projects/1279a8705bf445f68b6b41a695b6cf8c => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:29:24.936956 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4ed27-56dd-4283-b480-b42dbc6342b6 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:24.937242 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4ed27-56dd-4283-b480-b42dbc6342b6 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:24.937518 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4ed27-56dd-4283-b480-b42dbc6342b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:24.961935 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-c3e4ed27-56dd-4283-b480-b42dbc6342b6 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:24.963149 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-c3e4ed27-56dd-4283-b480-b42dbc6342b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:25.003939 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:29:25.032062 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1099/2190] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:29:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:29:25.040808 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc19d054-1156-4f8b-83ba-481f4049cad5 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:25.041216 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc19d054-1156-4f8b-83ba-481f4049cad5 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:25.041502 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc19d054-1156-4f8b-83ba-481f4049cad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:25.064988 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-cc19d054-1156-4f8b-83ba-481f4049cad5 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:25.066451 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-cc19d054-1156-4f8b-83ba-481f4049cad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:25.079276 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1092/2191] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:29:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:29:25.086385 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4ed27-56dd-4283-b480-b42dbc6342b6 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:25.101409 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4ed27-56dd-4283-b480-b42dbc6342b6 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:25.106874 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84a662-ffd4-4c7d-aa96-4277027af2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:25.107473 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84a662-ffd4-4c7d-aa96-4277027af2c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:25.107746 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84a662-ffd4-4c7d-aa96-4277027af2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:25.108046 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84a662-ffd4-4c7d-aa96-4277027af2c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:25.108723 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb84a662-ffd4-4c7d-aa96-4277027af2c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:29:25.109017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb84a662-ffd4-4c7d-aa96-4277027af2c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:25.110621 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb84a662-ffd4-4c7d-aa96-4277027af2c2 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:25.122957 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1100/2192] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:29:25.127657 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc19d054-1156-4f8b-83ba-481f4049cad5 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:25.139381 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc19d054-1156-4f8b-83ba-481f4049cad5 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:25.144899 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6a121-729a-40d5-b1ff-a7a018b3af60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:25.145561 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6a121-729a-40d5-b1ff-a7a018b3af60 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:25.145842 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6a121-729a-40d5-b1ff-a7a018b3af60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:25.146116 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6a121-729a-40d5-b1ff-a7a018b3af60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:25.146806 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f6a121-729a-40d5-b1ff-a7a018b3af60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:25.148858 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f6a121-729a-40d5-b1ff-a7a018b3af60 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:25.184373 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1093/2193] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:29:25 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:29:25.192249 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84a662-ffd4-4c7d-aa96-4277027af2c2 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:25.204079 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84a662-ffd4-4c7d-aa96-4277027af2c2 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:25.272173 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aaae30-7fc3-4142-9a53-45ac52edf0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:25.273110 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aaae30-7fc3-4142-9a53-45ac52edf0a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:25.273574 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aaae30-7fc3-4142-9a53-45ac52edf0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:25.273952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aaae30-7fc3-4142-9a53-45ac52edf0a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:25.274615 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aaae30-7fc3-4142-9a53-45ac52edf0a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:25.276598 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aaae30-7fc3-4142-9a53-45ac52edf0a2 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:25.303079 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1101/2194] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:29:25 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:29:25.308618 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6a121-729a-40d5-b1ff-a7a018b3af60 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:25.320562 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6a121-729a-40d5-b1ff-a7a018b3af60 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:25.325370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf6744-b2cc-4175-aca1-179584148380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:25.325991 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf6744-b2cc-4175-aca1-179584148380 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:25.326270 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf6744-b2cc-4175-aca1-179584148380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:25.326543 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf6744-b2cc-4175-aca1-179584148380 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:25.327232 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cf6744-b2cc-4175-aca1-179584148380 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:25.328876 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cf6744-b2cc-4175-aca1-179584148380 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:25.339887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1094/2195] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:29:25.344791 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aaae30-7fc3-4142-9a53-45ac52edf0a2 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:25.356846 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aaae30-7fc3-4142-9a53-45ac52edf0a2 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:25.363167 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735251f3-661a-4a3c-9acb-ea10f703b4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:25.364931 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735251f3-661a-4a3c-9acb-ea10f703b4fc admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:25.365214 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735251f3-661a-4a3c-9acb-ea10f703b4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:25.365485 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735251f3-661a-4a3c-9acb-ea10f703b4fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:25.366186 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735251f3-661a-4a3c-9acb-ea10f703b4fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:25.368424 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735251f3-661a-4a3c-9acb-ea10f703b4fc admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:25.382283 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1102/2196] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:29:25.388438 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf6744-b2cc-4175-aca1-179584148380 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:25.404152 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf6744-b2cc-4175-aca1-179584148380 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:25.410654 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8086e-3926-4b46-8d16-591220ebb710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:25.411311 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8086e-3926-4b46-8d16-591220ebb710 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:25.411592 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8086e-3926-4b46-8d16-591220ebb710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:25.411854 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8086e-3926-4b46-8d16-591220ebb710 admin admin] PATH_INFO: `/v3/projects/4bb6fbda74a549049efd9f34871378e6/users/7aed455ca6074c5da5d19743b853362d/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:25.432800 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e8086e-3926-4b46-8d16-591220ebb710 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bb6fbda74a549049efd9f34871378e6, user_id=7aed455ca6074c5da5d19743b853362d, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:25.435708 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e8086e-3926-4b46-8d16-591220ebb710 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:25.452112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1095/2197] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:29:25 2022] PUT /identity/v3/projects/4bb6fbda74a549049efd9f34871378e6/users/7aed455ca6074c5da5d19743b853362d/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:29:25.466679 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b63b5-33d9-4b0d-8417-fc97f59bce72 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:25.466963 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b63b5-33d9-4b0d-8417-fc97f59bce72 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:25.467259 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b63b5-33d9-4b0d-8417-fc97f59bce72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:25.499503 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-b67b63b5-33d9-4b0d-8417-fc97f59bce72 None None] MFA Rules not processed for user `7aed455ca6074c5da5d19743b853362d`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:25.500774 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-b67b63b5-33d9-4b0d-8417-fc97f59bce72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:25.591004 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1103/2198] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:29:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:29:25.596216 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8086e-3926-4b46-8d16-591220ebb710 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:25.665377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:29:25.684183 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8086e-3926-4b46-8d16-591220ebb710 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:25.689272 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce7e81-4cd8-4b47-86b9-2bb260f3c132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:25.689910 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce7e81-4cd8-4b47-86b9-2bb260f3c132 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:25.690198 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce7e81-4cd8-4b47-86b9-2bb260f3c132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:25.690467 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce7e81-4cd8-4b47-86b9-2bb260f3c132 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:25.691201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ce7e81-4cd8-4b47-86b9-2bb260f3c132 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:25.693517 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ce7e81-4cd8-4b47-86b9-2bb260f3c132 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:25.735309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1096/2199] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:29:25 2022] POST /identity/v3/projects => generated 371 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:29:25.740167 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b63b5-33d9-4b0d-8417-fc97f59bce72 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:25.752844 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b63b5-33d9-4b0d-8417-fc97f59bce72 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:25.796330 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:29:25.829171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cafb1-6720-4e3f-99f8-2e49177eb31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:25.829810 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cafb1-6720-4e3f-99f8-2e49177eb31a admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:25.830099 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cafb1-6720-4e3f-99f8-2e49177eb31a admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:25.830399 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cafb1-6720-4e3f-99f8-2e49177eb31a admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:25.831008 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23cafb1-6720-4e3f-99f8-2e49177eb31a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:25.833378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23cafb1-6720-4e3f-99f8-2e49177eb31a admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:25.862348 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1104/2200] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:29:25 2022] POST /identity/v3/users => generated 292 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:29:25.867169 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce7e81-4cd8-4b47-86b9-2bb260f3c132 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:25.882577 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce7e81-4cd8-4b47-86b9-2bb260f3c132 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:25.888078 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ac8e3-177f-4ad5-946c-3126f3495d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:25.888796 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ac8e3-177f-4ad5-946c-3126f3495d60 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:25.889301 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ac8e3-177f-4ad5-946c-3126f3495d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:25.889759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ac8e3-177f-4ad5-946c-3126f3495d60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:25.890860 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85ac8e3-177f-4ad5-946c-3126f3495d60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:25.892365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85ac8e3-177f-4ad5-946c-3126f3495d60 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:25.904412 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1097/2201] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:29:25.908949 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cafb1-6720-4e3f-99f8-2e49177eb31a admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:25.920308 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cafb1-6720-4e3f-99f8-2e49177eb31a admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:25.928242 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d3dc2-805d-4633-b616-e151369bb9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:25.929243 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d3dc2-805d-4633-b616-e151369bb9c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:25.929630 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d3dc2-805d-4633-b616-e151369bb9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:25.930027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d3dc2-805d-4633-b616-e151369bb9c1 admin admin] PATH_INFO: `/v3/projects/2de1c34934ce46fa80acd1e8e21e895f/users/cc55df12a9624aad8567a5d48f427012/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:25.952962 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d3dc2-805d-4633-b616-e151369bb9c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2de1c34934ce46fa80acd1e8e21e895f, user_id=cc55df12a9624aad8567a5d48f427012, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:25.960430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d3dc2-805d-4633-b616-e151369bb9c1 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:25.985650 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1105/2202] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:29:25 2022] PUT /identity/v3/projects/2de1c34934ce46fa80acd1e8e21e895f/users/cc55df12a9624aad8567a5d48f427012/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:29:25.989914 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ac8e3-177f-4ad5-946c-3126f3495d60 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:26.003877 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ac8e3-177f-4ad5-946c-3126f3495d60 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:26.079364 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fefd16-a89a-452d-a92f-6cb69fa6b55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:26.080092 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fefd16-a89a-452d-a92f-6cb69fa6b55a admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:26.080374 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fefd16-a89a-452d-a92f-6cb69fa6b55a admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:26.080646 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fefd16-a89a-452d-a92f-6cb69fa6b55a admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:26.081319 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fefd16-a89a-452d-a92f-6cb69fa6b55a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:26.083011 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fefd16-a89a-452d-a92f-6cb69fa6b55a admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:26.093887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1098/2203] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:29:26.099156 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d3dc2-805d-4633-b616-e151369bb9c1 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:26.111860 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d3dc2-805d-4633-b616-e151369bb9c1 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:26.117298 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5ace9-c5ba-4dd2-9ec4-c3a29d824cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:26.117888 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5ace9-c5ba-4dd2-9ec4-c3a29d824cf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:26.118163 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5ace9-c5ba-4dd2-9ec4-c3a29d824cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:26.118432 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5ace9-c5ba-4dd2-9ec4-c3a29d824cf2 admin admin] PATH_INFO: `/v3/projects/2de1c34934ce46fa80acd1e8e21e895f/users/cc55df12a9624aad8567a5d48f427012/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:26.120462 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e5ace9-c5ba-4dd2-9ec4-c3a29d824cf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2de1c34934ce46fa80acd1e8e21e895f, user_id=cc55df12a9624aad8567a5d48f427012, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:26.123478 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e5ace9-c5ba-4dd2-9ec4-c3a29d824cf2 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:26.135584 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1106/2204] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:29:26 2022] PUT /identity/v3/projects/2de1c34934ce46fa80acd1e8e21e895f/users/cc55df12a9624aad8567a5d48f427012/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:29:26.147065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e34e3-8fbf-4460-bee1-db18cc1b28e2 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:26.147350 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e34e3-8fbf-4460-bee1-db18cc1b28e2 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:26.147625 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e34e3-8fbf-4460-bee1-db18cc1b28e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:26.195215 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-cb5e34e3-8fbf-4460-bee1-db18cc1b28e2 None None] MFA Rules not processed for user `cc55df12a9624aad8567a5d48f427012`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:26.196501 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-cb5e34e3-8fbf-4460-bee1-db18cc1b28e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:26.274700 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1099/2205] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:29:26 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:29:26.286048 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5ace9-c5ba-4dd2-9ec4-c3a29d824cf2 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:26.288365 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-38e5ace9-c5ba-4dd2-9ec4-c3a29d824cf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:26.361748 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5ace9-c5ba-4dd2-9ec4-c3a29d824cf2 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:26.366140 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a6124-c1b7-45f6-92cb-1c382418732f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:26.366741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a6124-c1b7-45f6-92cb-1c382418732f service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:26.367016 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a6124-c1b7-45f6-92cb-1c382418732f service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:26.367282 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a6124-c1b7-45f6-92cb-1c382418732f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:26.437665 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a6124-c1b7-45f6-92cb-1c382418732f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:26.439257 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a6124-c1b7-45f6-92cb-1c382418732f service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:26.448804 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1107/2206] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:29:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:29:29.050526 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e34e3-8fbf-4460-bee1-db18cc1b28e2 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:29.100209 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:29:29.119444 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e34e3-8fbf-4460-bee1-db18cc1b28e2 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:29.124049 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd70bf-4def-4967-9857-2d1efcd6beb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:29.124692 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd70bf-4def-4967-9857-2d1efcd6beb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:29.124974 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd70bf-4def-4967-9857-2d1efcd6beb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:29.125249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd70bf-4def-4967-9857-2d1efcd6beb9 admin admin] PATH_INFO: `/v3/users/7aed455ca6074c5da5d19743b853362d` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:29.126387 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd70bf-4def-4967-9857-2d1efcd6beb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7aed455ca6074c5da5d19743b853362d)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:29.128382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd70bf-4def-4967-9857-2d1efcd6beb9 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:29.179807 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f6fd70bf-4def-4967-9857-2d1efcd6beb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7aed455ca6074c5da5d19743b853362d'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:29.188777 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f6fd70bf-4def-4967-9857-2d1efcd6beb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7aed455ca6074c5da5d19743b853362d'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:29.202593 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f6fd70bf-4def-4967-9857-2d1efcd6beb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7aed455ca6074c5da5d19743b853362d'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:29.203287 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f6fd70bf-4def-4967-9857-2d1efcd6beb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7aed455ca6074c5da5d19743b853362d'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:29.210370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f6fd70bf-4def-4967-9857-2d1efcd6beb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7aed455ca6074c5da5d19743b853362d'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:29.217029 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1100/2207] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:29:29 2022] DELETE /identity/v3/users/7aed455ca6074c5da5d19743b853362d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:29:29.221445 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e24aa5-787e-4741-9da3-92e56557194f None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:29.222014 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e24aa5-787e-4741-9da3-92e56557194f None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:29.222014 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e24aa5-787e-4741-9da3-92e56557194f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:29.241513 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-89e24aa5-787e-4741-9da3-92e56557194f None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:29.242770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-89e24aa5-787e-4741-9da3-92e56557194f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:29.283597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:29:29.309102 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1108/2208] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:29:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:29:29.320587 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd70bf-4def-4967-9857-2d1efcd6beb9 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:29.322453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-f6fd70bf-4def-4967-9857-2d1efcd6beb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:29.388121 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd70bf-4def-4967-9857-2d1efcd6beb9 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:29.394087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8e222-b127-4fee-a5f7-ca667349e586 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:29.394087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8e222-b127-4fee-a5f7-ca667349e586 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:29.394087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8e222-b127-4fee-a5f7-ca667349e586 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:29.394087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8e222-b127-4fee-a5f7-ca667349e586 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:29.412993 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c8e222-b127-4fee-a5f7-ca667349e586 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:29.414498 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c8e222-b127-4fee-a5f7-ca667349e586 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:29.424269 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1101/2209] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:29:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:29:29.948328 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e24aa5-787e-4741-9da3-92e56557194f None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:29.949833 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-89e24aa5-787e-4741-9da3-92e56557194f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:29.962299 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e24aa5-787e-4741-9da3-92e56557194f None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:29.967084 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46770847-80a2-4b7e-8b41-1838aceed936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:29.967832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46770847-80a2-4b7e-8b41-1838aceed936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:29.968217 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46770847-80a2-4b7e-8b41-1838aceed936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:29.968554 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46770847-80a2-4b7e-8b41-1838aceed936 admin admin] PATH_INFO: `/v3/projects/4bb6fbda74a549049efd9f34871378e6` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:29.969748 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46770847-80a2-4b7e-8b41-1838aceed936 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bb6fbda74a549049efd9f34871378e6)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:29.971733 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46770847-80a2-4b7e-8b41-1838aceed936 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:30.025591 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-46770847-80a2-4b7e-8b41-1838aceed936 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bb6fbda74a549049efd9f34871378e6', 'request_id': 'req-46770847-80a2-4b7e-8b41-1838aceed936'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:30.031939 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-46770847-80a2-4b7e-8b41-1838aceed936 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bb6fbda74a549049efd9f34871378e6', 'request_id': 'req-46770847-80a2-4b7e-8b41-1838aceed936'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:30.041194 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-46770847-80a2-4b7e-8b41-1838aceed936 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bb6fbda74a549049efd9f34871378e6', 'request_id': 'req-46770847-80a2-4b7e-8b41-1838aceed936'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:30.054410 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-46770847-80a2-4b7e-8b41-1838aceed936 admin admin] The token cache is being invalidate because project 4bb6fbda74a549049efd9f34871378e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:29:30.054755 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-46770847-80a2-4b7e-8b41-1838aceed936 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:30.055506 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-46770847-80a2-4b7e-8b41-1838aceed936 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:30.057369 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1109/2210] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:29:29 2022] DELETE /identity/v3/projects/4bb6fbda74a549049efd9f34871378e6 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:29:30.061659 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8e222-b127-4fee-a5f7-ca667349e586 service neutron] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:30.073582 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8e222-b127-4fee-a5f7-ca667349e586 service neutron] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:30.140722 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93733c2-02a1-414c-b385-a8d20e8e1796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:30.141068 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93733c2-02a1-414c-b385-a8d20e8e1796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:30.141350 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93733c2-02a1-414c-b385-a8d20e8e1796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:30.141626 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93733c2-02a1-414c-b385-a8d20e8e1796 admin admin] PATH_INFO: `/v3/users/cc55df12a9624aad8567a5d48f427012` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:30.142774 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93733c2-02a1-414c-b385-a8d20e8e1796 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc55df12a9624aad8567a5d48f427012)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:30.145006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93733c2-02a1-414c-b385-a8d20e8e1796 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:30.151371 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:29:30.210195 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a93733c2-02a1-414c-b385-a8d20e8e1796 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc55df12a9624aad8567a5d48f427012'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:30.221428 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a93733c2-02a1-414c-b385-a8d20e8e1796 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc55df12a9624aad8567a5d48f427012'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:30.249662 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a93733c2-02a1-414c-b385-a8d20e8e1796 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc55df12a9624aad8567a5d48f427012'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:30.250559 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a93733c2-02a1-414c-b385-a8d20e8e1796 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc55df12a9624aad8567a5d48f427012'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:30.260812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a93733c2-02a1-414c-b385-a8d20e8e1796 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc55df12a9624aad8567a5d48f427012'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:30.271839 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1102/2211] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:29:30 2022] DELETE /identity/v3/users/cc55df12a9624aad8567a5d48f427012 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:29:30.644660 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46770847-80a2-4b7e-8b41-1838aceed936 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:30.647154 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-46770847-80a2-4b7e-8b41-1838aceed936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:30.666086 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46770847-80a2-4b7e-8b41-1838aceed936 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:30.739566 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b59c6c-91a5-4ac9-8010-f0bc7b586c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:30.740232 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b59c6c-91a5-4ac9-8010-f0bc7b586c27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:30.740526 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b59c6c-91a5-4ac9-8010-f0bc7b586c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:30.740802 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b59c6c-91a5-4ac9-8010-f0bc7b586c27 admin admin] PATH_INFO: `/v3/projects/2de1c34934ce46fa80acd1e8e21e895f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:30.741993 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b59c6c-91a5-4ac9-8010-f0bc7b586c27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2de1c34934ce46fa80acd1e8e21e895f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:30.743944 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b59c6c-91a5-4ac9-8010-f0bc7b586c27 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:30.794099 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a7b59c6c-91a5-4ac9-8010-f0bc7b586c27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2de1c34934ce46fa80acd1e8e21e895f', 'request_id': 'req-a7b59c6c-91a5-4ac9-8010-f0bc7b586c27'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:30.800839 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a7b59c6c-91a5-4ac9-8010-f0bc7b586c27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2de1c34934ce46fa80acd1e8e21e895f', 'request_id': 'req-a7b59c6c-91a5-4ac9-8010-f0bc7b586c27'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:30.809798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a7b59c6c-91a5-4ac9-8010-f0bc7b586c27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2de1c34934ce46fa80acd1e8e21e895f', 'request_id': 'req-a7b59c6c-91a5-4ac9-8010-f0bc7b586c27'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:30.817984 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:29:30.825796 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a7b59c6c-91a5-4ac9-8010-f0bc7b586c27 admin admin] The token cache is being invalidate because project 2de1c34934ce46fa80acd1e8e21e895f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:29:30.826092 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a7b59c6c-91a5-4ac9-8010-f0bc7b586c27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:30.827065 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a7b59c6c-91a5-4ac9-8010-f0bc7b586c27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:29:30.828787 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1110/2212] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:29:30 2022] DELETE /identity/v3/projects/2de1c34934ce46fa80acd1e8e21e895f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:29:35.265303 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a356a0-1f7b-4773-9432-672c41da7a57 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:35.265776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a356a0-1f7b-4773-9432-672c41da7a57 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:35.265871 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a356a0-1f7b-4773-9432-672c41da7a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:35.286815 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-70a356a0-1f7b-4773-9432-672c41da7a57 None None] MFA Rules not processed for user `d3538271ebfd41aabe9f3da85feb7bca`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:35.288066 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-70a356a0-1f7b-4773-9432-672c41da7a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:35.348076 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1103/2213] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Aug 23 04:29:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:29:35.360624 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b59c6c-91a5-4ac9-8010-f0bc7b586c27 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:35.362737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a7b59c6c-91a5-4ac9-8010-f0bc7b586c27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:35.439848 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b59c6c-91a5-4ac9-8010-f0bc7b586c27 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:35.445460 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b7ac8-6539-43fb-93f9-7756c5982ca9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f1418fd211a448da1a50175dce13b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:35.446107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b7ac8-6539-43fb-93f9-7756c5982ca9 service swift] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:35.446383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b7ac8-6539-43fb-93f9-7756c5982ca9 service swift] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:35.446652 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b7ac8-6539-43fb-93f9-7756c5982ca9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:35.455576 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b7ac8-6539-43fb-93f9-7756c5982ca9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:35.457254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b7ac8-6539-43fb-93f9-7756c5982ca9 service swift] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:35.475349 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1111/2214] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Aug 23 04:29:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 04:29:45.340635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=48,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:29:45.464188 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=33 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:29:50.009341 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38504a39-8597-460f-a9b9-01e6a711626c None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:50.009884 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38504a39-8597-460f-a9b9-01e6a711626c None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:50.009884 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38504a39-8597-460f-a9b9-01e6a711626c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:50.040444 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-38504a39-8597-460f-a9b9-01e6a711626c None None] MFA Rules not processed for user `4eb4a476712c492da2313ef426001e33`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:50.042138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-38504a39-8597-460f-a9b9-01e6a711626c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:50.114180 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1104/2215] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:29:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:29:51.847439 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387074d-34fe-42bb-bb34-2c857ae7d89f None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:51.848320 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387074d-34fe-42bb-bb34-2c857ae7d89f None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:51.848589 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387074d-34fe-42bb-bb34-2c857ae7d89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:51.868114 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-9387074d-34fe-42bb-bb34-2c857ae7d89f None None] MFA Rules not processed for user `a91872508ecd49d98bd95651412f85a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:51.869329 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-9387074d-34fe-42bb-bb34-2c857ae7d89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:51.930785 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1112/2216] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:29:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:29:54.864802 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45e7b4-d608-498e-9fdf-0c4ad1fa6856 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:54.865847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45e7b4-d608-498e-9fdf-0c4ad1fa6856 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:54.866572 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45e7b4-d608-498e-9fdf-0c4ad1fa6856 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:54.868605 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1105/2217] 10.222.0.18 () {54 vars in 921 bytes} [Tue Aug 23 04:29:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:29:54.874645 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412e275-a34e-485b-87cb-07551339b5c3 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:54.875140 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412e275-a34e-485b-87cb-07551339b5c3 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:54.875362 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412e275-a34e-485b-87cb-07551339b5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:54.899852 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-e412e275-a34e-485b-87cb-07551339b5c3 None None] MFA Rules not processed for user `cd7b11f479a54adc8d9b8641859b07a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:54.901106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-e412e275-a34e-485b-87cb-07551339b5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:54.981288 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1113/2218] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Aug 23 04:29:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:29:54.994454 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45e7b4-d608-498e-9fdf-0c4ad1fa6856 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:54.996204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-de45e7b4-d608-498e-9fdf-0c4ad1fa6856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:55.078322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45e7b4-d608-498e-9fdf-0c4ad1fa6856 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:55.082027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e36633-5ef1-43c0-a877-c13ed9869b9b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f1bf93cff6c46e0801bb827203cb5ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:55.082853 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e36633-5ef1-43c0-a877-c13ed9869b9b service glance] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:55.083133 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e36633-5ef1-43c0-a877-c13ed9869b9b service glance] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:55.083430 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e36633-5ef1-43c0-a877-c13ed9869b9b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:55.095170 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e36633-5ef1-43c0-a877-c13ed9869b9b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:55.096678 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e36633-5ef1-43c0-a877-c13ed9869b9b service glance] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:55.108927 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1106/2219] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:29:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:29:55.178497 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c43a0-7233-4dde-ab02-ae5ee02f502e None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:55.178822 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c43a0-7233-4dde-ab02-ae5ee02f502e None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:55.179131 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c43a0-7233-4dde-ab02-ae5ee02f502e None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:55.180754 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1114/2220] 10.222.0.18 () {54 vars in 921 bytes} [Tue Aug 23 04:29:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:29:55.185640 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a37963-a8e0-4322-80b0-9ef3cd5a0e2b None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:55.185945 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a37963-a8e0-4322-80b0-9ef3cd5a0e2b None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:55.186218 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a37963-a8e0-4322-80b0-9ef3cd5a0e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:55.207237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-94a37963-a8e0-4322-80b0-9ef3cd5a0e2b None None] MFA Rules not processed for user `fdca9c48e5d740f6bda8c48072b54c0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:55.208875 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-94a37963-a8e0-4322-80b0-9ef3cd5a0e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:55.268433 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1107/2221] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Aug 23 04:29:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:29:55.280754 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c43a0-7233-4dde-ab02-ae5ee02f502e None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:55.284112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-498c43a0-7233-4dde-ab02-ae5ee02f502e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:55.298855 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c43a0-7233-4dde-ab02-ae5ee02f502e None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:55.302857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496a7bd-1518-4ebe-a1e7-8a54ee89bdea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:55.303669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496a7bd-1518-4ebe-a1e7-8a54ee89bdea service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:55.304020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496a7bd-1518-4ebe-a1e7-8a54ee89bdea service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:55.304302 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496a7bd-1518-4ebe-a1e7-8a54ee89bdea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:55.314211 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4496a7bd-1518-4ebe-a1e7-8a54ee89bdea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:55.315674 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4496a7bd-1518-4ebe-a1e7-8a54ee89bdea service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:55.325235 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1115/2222] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:29:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:29:55.416609 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b613e2e-0e47-4a93-9ee8-9b747ef18a22 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:55.416895 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b613e2e-0e47-4a93-9ee8-9b747ef18a22 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:55.417170 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b613e2e-0e47-4a93-9ee8-9b747ef18a22 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:55.418790 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1108/2223] 10.222.0.18 () {54 vars in 921 bytes} [Tue Aug 23 04:29:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:29:55.424518 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7f4f8-a85a-4e7d-a552-723a323e77a1 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:55.424951 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7f4f8-a85a-4e7d-a552-723a323e77a1 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:55.425227 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7f4f8-a85a-4e7d-a552-723a323e77a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:55.444512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-dbf7f4f8-a85a-4e7d-a552-723a323e77a1 None None] MFA Rules not processed for user `cd7b11f479a54adc8d9b8641859b07a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:29:55.445763 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-dbf7f4f8-a85a-4e7d-a552-723a323e77a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:29:55.455417 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1116/2224] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Aug 23 04:29:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:29:55.470238 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b613e2e-0e47-4a93-9ee8-9b747ef18a22 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:29:55.483111 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b613e2e-0e47-4a93-9ee8-9b747ef18a22 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:29:55.487957 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38057f52-e04a-4925-aa83-3a4a8fd60ca1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:29:55.489181 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38057f52-e04a-4925-aa83-3a4a8fd60ca1 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:29:55.489181 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38057f52-e04a-4925-aa83-3a4a8fd60ca1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:29:55.489623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38057f52-e04a-4925-aa83-3a4a8fd60ca1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:29:55.502295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38057f52-e04a-4925-aa83-3a4a8fd60ca1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:29:55.504008 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38057f52-e04a-4925-aa83-3a4a8fd60ca1 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:29:55.516310 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1109/2225] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:29:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:30:00.480436 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f6575-8acd-4786-acf3-cf1fe6b67276 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:30:00.480996 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f6575-8acd-4786-acf3-cf1fe6b67276 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:30:00.481111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f6575-8acd-4786-acf3-cf1fe6b67276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:30:00.500532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-bc7f6575-8acd-4786-acf3-cf1fe6b67276 None None] MFA Rules not processed for user `820b9a99be574eb8b09629988c4c59c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:30:00.501800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-bc7f6575-8acd-4786-acf3-cf1fe6b67276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:30:00.568153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1117/2226] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:30:00 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:30:00.854407 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0549f-955f-4958-8a61-97a736b0c5b8 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:30:00.854922 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0549f-955f-4958-8a61-97a736b0c5b8 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:30:00.854922 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0549f-955f-4958-8a61-97a736b0c5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:30:00.876120 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-28c0549f-955f-4958-8a61-97a736b0c5b8 None None] MFA Rules not processed for user `d3538271ebfd41aabe9f3da85feb7bca`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:30:00.877344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-28c0549f-955f-4958-8a61-97a736b0c5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:30:00.901079 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1110/2227] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Aug 23 04:30:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:30:00.921734 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f6575-8acd-4786-acf3-cf1fe6b67276 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:30:00.934185 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f6575-8acd-4786-acf3-cf1fe6b67276 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:30:00.940023 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc2626-055e-41fc-92af-e14617bb0549 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f1418fd211a448da1a50175dce13b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:30:00.940830 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc2626-055e-41fc-92af-e14617bb0549 service swift] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:30:00.941109 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc2626-055e-41fc-92af-e14617bb0549 service swift] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:30:00.941376 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc2626-055e-41fc-92af-e14617bb0549 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:30:00.950509 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc2626-055e-41fc-92af-e14617bb0549 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:30:00.952268 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc2626-055e-41fc-92af-e14617bb0549 service swift] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:30:00.972125 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1118/2228] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Aug 23 04:30:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 04:30:10.870849 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=92 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:30:10.958187 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=96 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:30:29.983355 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0549f-955f-4958-8a61-97a736b0c5b8 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:30:29.984820 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-28c0549f-955f-4958-8a61-97a736b0c5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:30:30.061575 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0549f-955f-4958-8a61-97a736b0c5b8 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:30:30.067015 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d8722-67d2-43f0-86a6-a0f2a9c620e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd503fe3cf6d428e962c14ab864ee608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:30:30.067776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d8722-67d2-43f0-86a6-a0f2a9c620e6 service placement] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:30:30.068108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d8722-67d2-43f0-86a6-a0f2a9c620e6 service placement] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:30:30.068443 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d8722-67d2-43f0-86a6-a0f2a9c620e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:30:30.070403 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-7f7d8722-67d2-43f0-86a6-a0f2a9c620e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:30:30.082189 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d8722-67d2-43f0-86a6-a0f2a9c620e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:30:30.083943 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d8722-67d2-43f0-86a6-a0f2a9c620e6 service placement] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:30:30.094544 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1111/2229] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:30:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:30:31.421434 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a111e-50d2-4076-b263-87e08a866c1c None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:30:31.421943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a111e-50d2-4076-b263-87e08a866c1c None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:30:31.421943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a111e-50d2-4076-b263-87e08a866c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:30:31.441601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-976a111e-50d2-4076-b263-87e08a866c1c None None] MFA Rules not processed for user `d3538271ebfd41aabe9f3da85feb7bca`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:30:31.443302 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-976a111e-50d2-4076-b263-87e08a866c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:30:31.450604 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1119/2230] 10.222.0.18 () {58 vars in 967 bytes} [Tue Aug 23 04:30:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:30:31.473993 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d8722-67d2-43f0-86a6-a0f2a9c620e6 service placement] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:30:31.491519 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d8722-67d2-43f0-86a6-a0f2a9c620e6 service placement] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:30:31.497593 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12b065-ec45-4b9a-943a-1a83d1a0ed1a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f1418fd211a448da1a50175dce13b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:30:31.498204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12b065-ec45-4b9a-943a-1a83d1a0ed1a service swift] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:30:31.498482 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12b065-ec45-4b9a-943a-1a83d1a0ed1a service swift] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:30:31.498751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12b065-ec45-4b9a-943a-1a83d1a0ed1a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:30:31.508669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12b065-ec45-4b9a-943a-1a83d1a0ed1a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:30:31.510436 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12b065-ec45-4b9a-943a-1a83d1a0ed1a service swift] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:30:31.518948 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1112/2231] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Aug 23 04:30:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 04:30:41.436813 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:30:41.515668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=38 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:31:04.692246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a111e-50d2-4076-b263-87e08a866c1c None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:31:04.693866 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-976a111e-50d2-4076-b263-87e08a866c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:31:04.762862 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a111e-50d2-4076-b263-87e08a866c1c None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:31:04.767747 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7af580-2563-433f-bc20-02c2d60fb5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:31:04.768405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7af580-2563-433f-bc20-02c2d60fb5bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:31:04.768684 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7af580-2563-433f-bc20-02c2d60fb5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:31:04.768955 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7af580-2563-433f-bc20-02c2d60fb5bd admin admin] PATH_INFO: `/v3/users/4eb4a476712c492da2313ef426001e33` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:31:04.770246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7af580-2563-433f-bc20-02c2d60fb5bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4eb4a476712c492da2313ef426001e33)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:31:04.772668 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7af580-2563-433f-bc20-02c2d60fb5bd admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:31:04.822328 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-bb7af580-2563-433f-bc20-02c2d60fb5bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4eb4a476712c492da2313ef426001e33'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:31:04.823026 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-bb7af580-2563-433f-bc20-02c2d60fb5bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4eb4a476712c492da2313ef426001e33'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:31:04.831740 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-bb7af580-2563-433f-bc20-02c2d60fb5bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4eb4a476712c492da2313ef426001e33'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:31:04.844735 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-bb7af580-2563-433f-bc20-02c2d60fb5bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4eb4a476712c492da2313ef426001e33'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:31:04.852931 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-bb7af580-2563-433f-bc20-02c2d60fb5bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4eb4a476712c492da2313ef426001e33'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:31:04.859272 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1120/2232] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:31:04 2022] DELETE /identity/v3/users/4eb4a476712c492da2313ef426001e33 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:31:05.036591 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12b065-ec45-4b9a-943a-1a83d1a0ed1a service swift] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:31:05.048761 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12b065-ec45-4b9a-943a-1a83d1a0ed1a service swift] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:31:05.111625 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e9e704-7187-4cc2-936f-276b56f9c927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:31:05.112569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9e704-7187-4cc2-936f-276b56f9c927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:31:05.112960 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9e704-7187-4cc2-936f-276b56f9c927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:31:05.113383 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9e704-7187-4cc2-936f-276b56f9c927 admin admin] PATH_INFO: `/v3/projects/57d3d5a2a0514d7687e794f8a678cff9` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:31:05.114906 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e9e704-7187-4cc2-936f-276b56f9c927 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57d3d5a2a0514d7687e794f8a678cff9)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:31:05.117748 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e9e704-7187-4cc2-936f-276b56f9c927 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:31:05.171411 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-27e9e704-7187-4cc2-936f-276b56f9c927 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57d3d5a2a0514d7687e794f8a678cff9', 'request_id': 'req-27e9e704-7187-4cc2-936f-276b56f9c927'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:31:05.178163 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-27e9e704-7187-4cc2-936f-276b56f9c927 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57d3d5a2a0514d7687e794f8a678cff9', 'request_id': 'req-27e9e704-7187-4cc2-936f-276b56f9c927'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:31:05.186335 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-27e9e704-7187-4cc2-936f-276b56f9c927 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57d3d5a2a0514d7687e794f8a678cff9', 'request_id': 'req-27e9e704-7187-4cc2-936f-276b56f9c927'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:31:05.201210 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-27e9e704-7187-4cc2-936f-276b56f9c927 admin admin] The token cache is being invalidate because project 57d3d5a2a0514d7687e794f8a678cff9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:31:05.201499 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-27e9e704-7187-4cc2-936f-276b56f9c927 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:31:05.202223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-27e9e704-7187-4cc2-936f-276b56f9c927 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:31:05.204458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1113/2233] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:31:05 2022] DELETE /identity/v3/projects/57d3d5a2a0514d7687e794f8a678cff9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:31:05.209584 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afea04c-b33b-439f-844d-a863d88ccd1a None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:31:05.209936 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afea04c-b33b-439f-844d-a863d88ccd1a None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:31:05.210239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afea04c-b33b-439f-844d-a863d88ccd1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:31:05.233687 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-3afea04c-b33b-439f-844d-a863d88ccd1a None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:31:05.234906 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-3afea04c-b33b-439f-844d-a863d88ccd1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:31:05.312067 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1121/2234] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:31:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:31:05.320143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d144566-a6ca-4a80-85f6-cf0f6f218618 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:31:05.320404 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d144566-a6ca-4a80-85f6-cf0f6f218618 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:31:05.320698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d144566-a6ca-4a80-85f6-cf0f6f218618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:31:05.341820 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-0d144566-a6ca-4a80-85f6-cf0f6f218618 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:31:05.343047 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-0d144566-a6ca-4a80-85f6-cf0f6f218618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:31:05.352714 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1114/2235] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:31:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:31:05.357169 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afea04c-b33b-439f-844d-a863d88ccd1a None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:31:05.367594 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afea04c-b33b-439f-844d-a863d88ccd1a None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:31:05.373936 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90844400-50d7-46c8-b568-3f74235dcdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:31:05.374534 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90844400-50d7-46c8-b568-3f74235dcdbb admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:31:05.374808 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90844400-50d7-46c8-b568-3f74235dcdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:31:05.375078 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90844400-50d7-46c8-b568-3f74235dcdbb admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:31:05.375740 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90844400-50d7-46c8-b568-3f74235dcdbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:31:05.376071 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90844400-50d7-46c8-b568-3f74235dcdbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:31:05.377666 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90844400-50d7-46c8-b568-3f74235dcdbb admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:31:05.390120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1122/2236] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:31:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:31:05.394463 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d144566-a6ca-4a80-85f6-cf0f6f218618 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:31:05.404400 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d144566-a6ca-4a80-85f6-cf0f6f218618 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:31:05.409752 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a9800e-94d7-4c70-ae28-58c44f5bcb22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:31:05.410369 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9800e-94d7-4c70-ae28-58c44f5bcb22 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:31:05.411187 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9800e-94d7-4c70-ae28-58c44f5bcb22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:31:05.411496 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9800e-94d7-4c70-ae28-58c44f5bcb22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:31:05.412154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a9800e-94d7-4c70-ae28-58c44f5bcb22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:31:05.414073 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a9800e-94d7-4c70-ae28-58c44f5bcb22 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:31:05.445435 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1115/2237] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:31:05 2022] POST /identity/v3/projects => generated 391 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:31:05.449533 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90844400-50d7-46c8-b568-3f74235dcdbb admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:31:05.464562 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90844400-50d7-46c8-b568-3f74235dcdbb admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:31:05.468533 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:31:05.531084 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c504b3a-fbad-4c1a-a5a1-6c7b4bfcbbb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:31:05.531703 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c504b3a-fbad-4c1a-a5a1-6c7b4bfcbbb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:31:05.532012 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c504b3a-fbad-4c1a-a5a1-6c7b4bfcbbb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:31:05.532290 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c504b3a-fbad-4c1a-a5a1-6c7b4bfcbbb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:31:05.532944 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c504b3a-fbad-4c1a-a5a1-6c7b4bfcbbb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:31:05.534927 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c504b3a-fbad-4c1a-a5a1-6c7b4bfcbbb3 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:31:05.558913 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1123/2238] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:31:05 2022] POST /identity/v3/users => generated 296 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:31:05.563111 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a9800e-94d7-4c70-ae28-58c44f5bcb22 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:31:05.574542 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a9800e-94d7-4c70-ae28-58c44f5bcb22 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:31:05.579960 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d6d65-2b7d-4e93-aa2c-0c2ae3453aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:31:05.580729 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d6d65-2b7d-4e93-aa2c-0c2ae3453aee admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:31:05.581013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d6d65-2b7d-4e93-aa2c-0c2ae3453aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:31:05.581284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d6d65-2b7d-4e93-aa2c-0c2ae3453aee admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:31:05.581938 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d6d65-2b7d-4e93-aa2c-0c2ae3453aee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:31:05.583693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d6d65-2b7d-4e93-aa2c-0c2ae3453aee admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:31:05.595280 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1116/2239] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:31:05.600275 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c504b3a-fbad-4c1a-a5a1-6c7b4bfcbbb3 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:31:05.610388 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c504b3a-fbad-4c1a-a5a1-6c7b4bfcbbb3 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:31:05.615453 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74842d41-e413-456b-9bfe-6fff982975db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:31:05.616109 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74842d41-e413-456b-9bfe-6fff982975db admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:31:05.616391 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74842d41-e413-456b-9bfe-6fff982975db admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:31:05.616648 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74842d41-e413-456b-9bfe-6fff982975db admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:31:05.617302 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74842d41-e413-456b-9bfe-6fff982975db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:31:05.618956 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74842d41-e413-456b-9bfe-6fff982975db admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:31:05.630193 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1124/2240] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:31:05.634449 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d6d65-2b7d-4e93-aa2c-0c2ae3453aee admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:31:05.644492 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d6d65-2b7d-4e93-aa2c-0c2ae3453aee admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:31:05.649369 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a86c7-87e4-44eb-af57-b97c07cf0431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:31:05.650024 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a86c7-87e4-44eb-af57-b97c07cf0431 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:31:05.650305 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a86c7-87e4-44eb-af57-b97c07cf0431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:31:05.650583 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a86c7-87e4-44eb-af57-b97c07cf0431 admin admin] PATH_INFO: `/v3/projects/c281900ca99e49cea5c410b6eab0f420/users/85a594d7d5fd4c5fb0ad883b4c127936/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:31:05.666050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a86c7-87e4-44eb-af57-b97c07cf0431 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c281900ca99e49cea5c410b6eab0f420, user_id=85a594d7d5fd4c5fb0ad883b4c127936, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:31:05.668766 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a86c7-87e4-44eb-af57-b97c07cf0431 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:31:05.683654 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1117/2241] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:31:05 2022] PUT /identity/v3/projects/c281900ca99e49cea5c410b6eab0f420/users/85a594d7d5fd4c5fb0ad883b4c127936/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:31:05.695193 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923819b-9c3a-4bed-a147-2817140c9d13 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:31:05.695708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923819b-9c3a-4bed-a147-2817140c9d13 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:31:05.695853 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923819b-9c3a-4bed-a147-2817140c9d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:31:05.727154 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-c923819b-9c3a-4bed-a147-2817140c9d13 None None] MFA Rules not processed for user `85a594d7d5fd4c5fb0ad883b4c127936`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:31:05.728420 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-c923819b-9c3a-4bed-a147-2817140c9d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:31:05.801759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1125/2242] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:31:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:31:05.812705 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a86c7-87e4-44eb-af57-b97c07cf0431 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:31:05.814568 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-188a86c7-87e4-44eb-af57-b97c07cf0431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:31:05.854643 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:31:05.884856 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a86c7-87e4-44eb-af57-b97c07cf0431 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:31:05.888753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1cad1f-b48c-4733-a004-78a046876989 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:31:05.889370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1cad1f-b48c-4733-a004-78a046876989 service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:31:05.889710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1cad1f-b48c-4733-a004-78a046876989 service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:31:05.889992 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1cad1f-b48c-4733-a004-78a046876989 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:31:05.899764 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1cad1f-b48c-4733-a004-78a046876989 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:31:05.901799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1cad1f-b48c-4733-a004-78a046876989 service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:31:05.915770 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1118/2243] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:31:05 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:31:15.798402 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=63,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:31:15.909256 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=14 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:31:24.404088 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923819b-9c3a-4bed-a147-2817140c9d13 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:31:24.470385 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923819b-9c3a-4bed-a147-2817140c9d13 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:31:24.476046 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f715abd-678f-4e4e-94da-ecf0d76f841f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:31:24.476737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f715abd-678f-4e4e-94da-ecf0d76f841f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:31:24.477036 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f715abd-678f-4e4e-94da-ecf0d76f841f admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:31:24.477292 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f715abd-678f-4e4e-94da-ecf0d76f841f admin admin] PATH_INFO: `/v3/users/85a594d7d5fd4c5fb0ad883b4c127936` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:31:24.478428 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f715abd-678f-4e4e-94da-ecf0d76f841f admin admin] RBAC: Authorizing `identity:delete_user(user_id=85a594d7d5fd4c5fb0ad883b4c127936)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:31:24.480401 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f715abd-678f-4e4e-94da-ecf0d76f841f admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:31:24.544339 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-2f715abd-678f-4e4e-94da-ecf0d76f841f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85a594d7d5fd4c5fb0ad883b4c127936'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:31:24.545257 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-2f715abd-678f-4e4e-94da-ecf0d76f841f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85a594d7d5fd4c5fb0ad883b4c127936'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:31:24.554614 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-2f715abd-678f-4e4e-94da-ecf0d76f841f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85a594d7d5fd4c5fb0ad883b4c127936'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:31:24.567191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-2f715abd-678f-4e4e-94da-ecf0d76f841f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85a594d7d5fd4c5fb0ad883b4c127936'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:31:24.574008 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-2f715abd-678f-4e4e-94da-ecf0d76f841f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85a594d7d5fd4c5fb0ad883b4c127936'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:31:24.580443 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1126/2244] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:31:24 2022] DELETE /identity/v3/users/85a594d7d5fd4c5fb0ad883b4c127936 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:31:24.585730 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcad2e6-49bf-437b-8e81-47399a79da56 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:31:24.586367 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcad2e6-49bf-437b-8e81-47399a79da56 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:31:24.586481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcad2e6-49bf-437b-8e81-47399a79da56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:31:24.606298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-5bcad2e6-49bf-437b-8e81-47399a79da56 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:31:24.607513 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-5bcad2e6-49bf-437b-8e81-47399a79da56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:31:24.673350 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1119/2245] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:31:24.683480 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f715abd-678f-4e4e-94da-ecf0d76f841f admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:31:24.685116 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-2f715abd-678f-4e4e-94da-ecf0d76f841f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:31:24.754220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f715abd-678f-4e4e-94da-ecf0d76f841f admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:31:24.758471 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636f30d-737f-4568-88be-63f97802c8d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:31:24.759080 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636f30d-737f-4568-88be-63f97802c8d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:31:24.759373 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636f30d-737f-4568-88be-63f97802c8d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:31:24.759629 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636f30d-737f-4568-88be-63f97802c8d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:31:24.769784 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f636f30d-737f-4568-88be-63f97802c8d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:31:24.771993 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f636f30d-737f-4568-88be-63f97802c8d4 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:31:24.785110 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1127/2246] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:31:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:31:25.100590 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcad2e6-49bf-437b-8e81-47399a79da56 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:31:25.102328 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-5bcad2e6-49bf-437b-8e81-47399a79da56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:31:25.116477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcad2e6-49bf-437b-8e81-47399a79da56 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:31:25.122326 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7cdda-fd07-4f3b-a08a-c213462bb08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:31:25.122997 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7cdda-fd07-4f3b-a08a-c213462bb08c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:31:25.123298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7cdda-fd07-4f3b-a08a-c213462bb08c admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:31:25.123571 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7cdda-fd07-4f3b-a08a-c213462bb08c admin admin] PATH_INFO: `/v3/projects/c281900ca99e49cea5c410b6eab0f420` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:31:25.124819 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe7cdda-fd07-4f3b-a08a-c213462bb08c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c281900ca99e49cea5c410b6eab0f420)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:31:25.126850 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe7cdda-fd07-4f3b-a08a-c213462bb08c admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:31:25.181709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-bbe7cdda-fd07-4f3b-a08a-c213462bb08c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c281900ca99e49cea5c410b6eab0f420', 'request_id': 'req-bbe7cdda-fd07-4f3b-a08a-c213462bb08c'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:31:25.189855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-bbe7cdda-fd07-4f3b-a08a-c213462bb08c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c281900ca99e49cea5c410b6eab0f420', 'request_id': 'req-bbe7cdda-fd07-4f3b-a08a-c213462bb08c'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:31:25.198589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-bbe7cdda-fd07-4f3b-a08a-c213462bb08c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c281900ca99e49cea5c410b6eab0f420', 'request_id': 'req-bbe7cdda-fd07-4f3b-a08a-c213462bb08c'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:31:25.211429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-bbe7cdda-fd07-4f3b-a08a-c213462bb08c admin admin] The token cache is being invalidate because project c281900ca99e49cea5c410b6eab0f420 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:31:25.211738 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-bbe7cdda-fd07-4f3b-a08a-c213462bb08c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:31:25.212572 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-bbe7cdda-fd07-4f3b-a08a-c213462bb08c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:31:25.214069 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1120/2247] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:31:25 2022] DELETE /identity/v3/projects/c281900ca99e49cea5c410b6eab0f420 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:31:31.426333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b34ff-bd18-4dd0-8f36-c5e4593dfabf None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:31:31.427387 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b34ff-bd18-4dd0-8f36-c5e4593dfabf None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:31:31.427682 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b34ff-bd18-4dd0-8f36-c5e4593dfabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:31:31.451417 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-dd3b34ff-bd18-4dd0-8f36-c5e4593dfabf None None] MFA Rules not processed for user `c8fa0859e5c14c53a5a2473c95efe152`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:31:31.455982 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-dd3b34ff-bd18-4dd0-8f36-c5e4593dfabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:31:31.479423 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:31:31.521175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1128/2248] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:31:31 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:31:33.186006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9057570-ccb2-4774-9ec5-a16078905b28 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:31:33.186578 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9057570-ccb2-4774-9ec5-a16078905b28 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:31:33.186578 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9057570-ccb2-4774-9ec5-a16078905b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:31:33.209088 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-f9057570-ccb2-4774-9ec5-a16078905b28 None None] MFA Rules not processed for user `a91872508ecd49d98bd95651412f85a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:31:33.210488 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-f9057570-ccb2-4774-9ec5-a16078905b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:31:33.299421 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1121/2249] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:31:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:31:41.513996 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=15 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:31:42.236827 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df03e3-ea43-4d65-80ab-4c572071be9b None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:31:42.237142 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df03e3-ea43-4d65-80ab-4c572071be9b None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:31:42.237418 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df03e3-ea43-4d65-80ab-4c572071be9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:31:42.256529 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-99df03e3-ea43-4d65-80ab-4c572071be9b None None] MFA Rules not processed for user `820b9a99be574eb8b09629988c4c59c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:31:42.257748 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-99df03e3-ea43-4d65-80ab-4c572071be9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:31:42.312865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1129/2250] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:31:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:31:43.287137 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:31:52.306128 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=25 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:31:53.819042 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0903eb9-1bb0-4e51-9eec-cc87d36bce7b None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:31:53.819634 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0903eb9-1bb0-4e51-9eec-cc87d36bce7b None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:31:53.819634 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0903eb9-1bb0-4e51-9eec-cc87d36bce7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:31:53.840076 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-e0903eb9-1bb0-4e51-9eec-cc87d36bce7b None None] MFA Rules not processed for user `658ce95360454ea1a96a22af310e5a4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:31:53.841267 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-e0903eb9-1bb0-4e51-9eec-cc87d36bce7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:31:53.901306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1122/2251] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:31:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:32:03.893473 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=25 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:32:25.636604 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df03e3-ea43-4d65-80ab-4c572071be9b None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:32:25.638360 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-99df03e3-ea43-4d65-80ab-4c572071be9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:32:25.712482 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df03e3-ea43-4d65-80ab-4c572071be9b None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:32:25.717546 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df4a40-53da-4017-95e4-c1d2cb544dfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:32:25.718156 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df4a40-53da-4017-95e4-c1d2cb544dfe service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:32:25.718432 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df4a40-53da-4017-95e4-c1d2cb544dfe service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:32:25.718698 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df4a40-53da-4017-95e4-c1d2cb544dfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:32:25.720861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a7df4a40-53da-4017-95e4-c1d2cb544dfe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:32:25.795100 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df4a40-53da-4017-95e4-c1d2cb544dfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:32:25.796602 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df4a40-53da-4017-95e4-c1d2cb544dfe service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:32:25.807587 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1130/2252] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:32:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:32:26.966888 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0903eb9-1bb0-4e51-9eec-cc87d36bce7b None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:32:26.968730 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-e0903eb9-1bb0-4e51-9eec-cc87d36bce7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:32:26.982425 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0903eb9-1bb0-4e51-9eec-cc87d36bce7b None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:32:26.986381 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284ac79-1640-4c79-b109-e1c2ee36a78d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:32:26.986989 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284ac79-1640-4c79-b109-e1c2ee36a78d service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:32:26.987262 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284ac79-1640-4c79-b109-e1c2ee36a78d service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:32:26.987531 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284ac79-1640-4c79-b109-e1c2ee36a78d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:32:26.989446 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-a284ac79-1640-4c79-b109-e1c2ee36a78d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:32:27.004325 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a284ac79-1640-4c79-b109-e1c2ee36a78d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:32:27.005765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a284ac79-1640-4c79-b109-e1c2ee36a78d service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:32:27.017379 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1123/2253] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:32:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:32:29.040973 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df4a40-53da-4017-95e4-c1d2cb544dfe service cinder] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:32:29.042756 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a7df4a40-53da-4017-95e4-c1d2cb544dfe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:32:29.058317 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df4a40-53da-4017-95e4-c1d2cb544dfe service cinder] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:32:29.066591 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9dd0cf-3307-4b60-92c5-e4f776ea4957 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:32:29.067210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9dd0cf-3307-4b60-92c5-e4f776ea4957 service nova] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:32:29.067507 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9dd0cf-3307-4b60-92c5-e4f776ea4957 service nova] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:32:29.067773 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9dd0cf-3307-4b60-92c5-e4f776ea4957 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:32:29.069902 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-0c9dd0cf-3307-4b60-92c5-e4f776ea4957 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:32:29.082136 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9dd0cf-3307-4b60-92c5-e4f776ea4957 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:32:29.083755 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9dd0cf-3307-4b60-92c5-e4f776ea4957 service nova] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:32:29.093253 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1131/2254] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:32:29 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:32:33.053960 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284ac79-1640-4c79-b109-e1c2ee36a78d service cinder] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:32:33.063842 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284ac79-1640-4c79-b109-e1c2ee36a78d service cinder] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:32:33.070613 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd7f557-6a09-4aec-a1ab-b246853f73d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:32:33.071518 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7f557-6a09-4aec-a1ab-b246853f73d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:32:33.071947 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7f557-6a09-4aec-a1ab-b246853f73d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:32:33.072302 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7f557-6a09-4aec-a1ab-b246853f73d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:32:33.075235 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-3bd7f557-6a09-4aec-a1ab-b246853f73d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:32:33.091337 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd7f557-6a09-4aec-a1ab-b246853f73d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:32:33.092822 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd7f557-6a09-4aec-a1ab-b246853f73d5 service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:32:33.102554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1124/2255] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:32:33 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:32:37.099291 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9dd0cf-3307-4b60-92c5-e4f776ea4957 service nova] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:32:37.109721 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9dd0cf-3307-4b60-92c5-e4f776ea4957 service nova] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:32:37.114164 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d434607-747a-4586-90b8-287d0bf0e755 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:32:37.114765 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d434607-747a-4586-90b8-287d0bf0e755 service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:32:37.115040 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d434607-747a-4586-90b8-287d0bf0e755 service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:32:37.115315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d434607-747a-4586-90b8-287d0bf0e755 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:32:37.117198 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-6d434607-747a-4586-90b8-287d0bf0e755 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:32:37.128550 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d434607-747a-4586-90b8-287d0bf0e755 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:32:37.129973 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d434607-747a-4586-90b8-287d0bf0e755 service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:32:37.139148 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1132/2256] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:32:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:32:43.097767 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=12 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:32:44.045708 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e57a1-04e6-4f39-915e-9e12d323198d None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:32:44.045997 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e57a1-04e6-4f39-915e-9e12d323198d None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:32:44.046268 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e57a1-04e6-4f39-915e-9e12d323198d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:32:44.066417 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-b37e57a1-04e6-4f39-915e-9e12d323198d None None] MFA Rules not processed for user `a91872508ecd49d98bd95651412f85a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:32:44.067616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-b37e57a1-04e6-4f39-915e-9e12d323198d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:32:44.076439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1125/2257] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:32:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:32:44.400423 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d434607-747a-4586-90b8-287d0bf0e755 service cinder] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:32:44.402708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-6d434607-747a-4586-90b8-287d0bf0e755 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:32:44.467934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d434607-747a-4586-90b8-287d0bf0e755 service cinder] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:32:44.472011 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9d33f-c89f-4a13-8b00-cacc342653c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:32:44.472623 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9d33f-c89f-4a13-8b00-cacc342653c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:32:44.472897 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9d33f-c89f-4a13-8b00-cacc342653c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:32:44.473167 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9d33f-c89f-4a13-8b00-cacc342653c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:32:44.475250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-9cd9d33f-c89f-4a13-8b00-cacc342653c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:32:44.485149 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9d33f-c89f-4a13-8b00-cacc342653c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:32:44.486579 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9d33f-c89f-4a13-8b00-cacc342653c5 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:32:44.494468 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1133/2258] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:32:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:32:44.793436 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e57a1-04e6-4f39-915e-9e12d323198d None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:32:44.794896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-b37e57a1-04e6-4f39-915e-9e12d323198d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:32:44.812059 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e57a1-04e6-4f39-915e-9e12d323198d None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:32:44.819891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebad10a-941b-4afb-9bda-288ab6327ef9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:32:44.820763 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebad10a-941b-4afb-9bda-288ab6327ef9 service nova] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:32:44.826382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebad10a-941b-4afb-9bda-288ab6327ef9 service nova] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:32:44.826702 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebad10a-941b-4afb-9bda-288ab6327ef9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:32:44.828930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-bebad10a-941b-4afb-9bda-288ab6327ef9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:32:44.829369 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9d33f-c89f-4a13-8b00-cacc342653c5 service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:32:44.831388 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-9cd9d33f-c89f-4a13-8b00-cacc342653c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:32:44.841655 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebad10a-941b-4afb-9bda-288ab6327ef9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:32:44.843258 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebad10a-941b-4afb-9bda-288ab6327ef9 service nova] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:32:44.861906 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1126/2259] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:32:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:32:44.865473 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:32:44.916069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9d33f-c89f-4a13-8b00-cacc342653c5 service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:32:44.921303 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e0446-da98-474e-bbc0-3083418f7803 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd503fe3cf6d428e962c14ab864ee608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:32:44.921986 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e0446-da98-474e-bbc0-3083418f7803 service placement] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:32:44.922335 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e0446-da98-474e-bbc0-3083418f7803 service placement] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:32:44.922676 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e0446-da98-474e-bbc0-3083418f7803 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:32:44.924673 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-027e0446-da98-474e-bbc0-3083418f7803 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:32:44.937484 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e0446-da98-474e-bbc0-3083418f7803 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:32:44.939114 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e0446-da98-474e-bbc0-3083418f7803 service placement] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:32:44.949829 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1134/2260] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:32:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:32:48.941087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebad10a-941b-4afb-9bda-288ab6327ef9 service nova] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:32:48.942716 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-bebad10a-941b-4afb-9bda-288ab6327ef9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:32:48.955597 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebad10a-941b-4afb-9bda-288ab6327ef9 service nova] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:32:48.961884 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53af54-be79-4148-bc58-8d7a6c229500 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:32:48.962490 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53af54-be79-4148-bc58-8d7a6c229500 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:32:48.962765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53af54-be79-4148-bc58-8d7a6c229500 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:32:48.963031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53af54-be79-4148-bc58-8d7a6c229500 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:32:48.964832 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-ce53af54-be79-4148-bc58-8d7a6c229500 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:32:48.975232 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce53af54-be79-4148-bc58-8d7a6c229500 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:32:48.976720 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce53af54-be79-4148-bc58-8d7a6c229500 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:32:48.986115 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1127/2261] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:32:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:32:52.276695 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27235c-9458-4a78-911b-64a1d8fde8e8 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:32:52.277194 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27235c-9458-4a78-911b-64a1d8fde8e8 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:32:52.277416 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27235c-9458-4a78-911b-64a1d8fde8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:32:52.296870 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-2e27235c-9458-4a78-911b-64a1d8fde8e8 None None] MFA Rules not processed for user `d3538271ebfd41aabe9f3da85feb7bca`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:32:52.298065 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-2e27235c-9458-4a78-911b-64a1d8fde8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:32:52.355348 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1135/2262] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Aug 23 04:32:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:32:52.418525 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53af54-be79-4148-bc58-8d7a6c229500 service neutron] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:32:52.420923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-ce53af54-be79-4148-bc58-8d7a6c229500 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:32:52.511201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53af54-be79-4148-bc58-8d7a6c229500 service neutron] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:32:52.517558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8133c46-f980-4066-bcc4-0075221a2999 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f1418fd211a448da1a50175dce13b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:32:52.518935 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8133c46-f980-4066-bcc4-0075221a2999 service swift] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:32:52.519281 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8133c46-f980-4066-bcc4-0075221a2999 service swift] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:32:52.519895 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8133c46-f980-4066-bcc4-0075221a2999 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:32:52.531206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8133c46-f980-4066-bcc4-0075221a2999 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:32:52.536869 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8133c46-f980-4066-bcc4-0075221a2999 service swift] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:32:52.546407 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1128/2263] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Aug 23 04:32:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 04:32:54.636885 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27235c-9458-4a78-911b-64a1d8fde8e8 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:32:54.647061 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27235c-9458-4a78-911b-64a1d8fde8e8 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:32:54.659334 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce838b-72c2-44ad-904e-2c390b6ee43e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:32:54.660020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce838b-72c2-44ad-904e-2c390b6ee43e service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:32:54.660294 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce838b-72c2-44ad-904e-2c390b6ee43e service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:32:54.660766 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce838b-72c2-44ad-904e-2c390b6ee43e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:32:54.667013 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-e4ce838b-72c2-44ad-904e-2c390b6ee43e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:32:54.848147 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ce838b-72c2-44ad-904e-2c390b6ee43e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:32:54.850263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ce838b-72c2-44ad-904e-2c390b6ee43e service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:32:54.886111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1136/2264] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:32:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:32:59.160739 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8133c46-f980-4066-bcc4-0075221a2999 service swift] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:32:59.182937 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8133c46-f980-4066-bcc4-0075221a2999 service swift] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:32:59.196850 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f7dc8-54b1-4b9d-8dd3-3003bc5ed4c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:32:59.197663 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f7dc8-54b1-4b9d-8dd3-3003bc5ed4c0 service nova] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:32:59.197948 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f7dc8-54b1-4b9d-8dd3-3003bc5ed4c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:32:59.198222 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f7dc8-54b1-4b9d-8dd3-3003bc5ed4c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:32:59.202172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-818f7dc8-54b1-4b9d-8dd3-3003bc5ed4c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:32:59.229766 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818f7dc8-54b1-4b9d-8dd3-3003bc5ed4c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:32:59.231379 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818f7dc8-54b1-4b9d-8dd3-3003bc5ed4c0 service nova] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:32:59.249531 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1129/2265] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:32:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:32:59.791703 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce838b-72c2-44ad-904e-2c390b6ee43e service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:32:59.799307 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f7dc8-54b1-4b9d-8dd3-3003bc5ed4c0 service nova] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:32:59.801297 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-818f7dc8-54b1-4b9d-8dd3-3003bc5ed4c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:32:59.821219 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce838b-72c2-44ad-904e-2c390b6ee43e service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:32:59.829308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f7dc8-54b1-4b9d-8dd3-3003bc5ed4c0 service nova] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:32:59.832460 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c358f2-37db-4ece-a7f5-7f4ce35e4591 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd503fe3cf6d428e962c14ab864ee608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:32:59.836171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c358f2-37db-4ece-a7f5-7f4ce35e4591 service placement] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:32:59.837132 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c358f2-37db-4ece-a7f5-7f4ce35e4591 service placement] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:32:59.837556 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c358f2-37db-4ece-a7f5-7f4ce35e4591 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:32:59.840620 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250bd3f-af1a-40c4-b0d1-a520b56fb864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:32:59.841731 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-e6c358f2-37db-4ece-a7f5-7f4ce35e4591 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:32:59.842578 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250bd3f-af1a-40c4-b0d1-a520b56fb864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:32:59.844272 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250bd3f-af1a-40c4-b0d1-a520b56fb864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:32:59.844700 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250bd3f-af1a-40c4-b0d1-a520b56fb864 admin admin] PATH_INFO: `/v3/users/820b9a99be574eb8b09629988c4c59c0` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:32:59.846691 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9250bd3f-af1a-40c4-b0d1-a520b56fb864 admin admin] RBAC: Authorizing `identity:delete_user(user_id=820b9a99be574eb8b09629988c4c59c0)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:32:59.852255 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9250bd3f-af1a-40c4-b0d1-a520b56fb864 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:32:59.868838 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c358f2-37db-4ece-a7f5-7f4ce35e4591 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:32:59.875669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c358f2-37db-4ece-a7f5-7f4ce35e4591 service placement] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:32:59.899073 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1137/2266] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:32:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:32:59.955514 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9250bd3f-af1a-40c4-b0d1-a520b56fb864 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '820b9a99be574eb8b09629988c4c59c0'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:32:59.970243 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9250bd3f-af1a-40c4-b0d1-a520b56fb864 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '820b9a99be574eb8b09629988c4c59c0'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:32:59.998571 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9250bd3f-af1a-40c4-b0d1-a520b56fb864 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '820b9a99be574eb8b09629988c4c59c0'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:00.000103 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9250bd3f-af1a-40c4-b0d1-a520b56fb864 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '820b9a99be574eb8b09629988c4c59c0'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:00.010210 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9250bd3f-af1a-40c4-b0d1-a520b56fb864 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '820b9a99be574eb8b09629988c4c59c0'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:00.018015 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1130/2267] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:32:59 2022] DELETE /identity/v3/users/820b9a99be574eb8b09629988c4c59c0 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:00.277505 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c358f2-37db-4ece-a7f5-7f4ce35e4591 service placement] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:00.291940 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c358f2-37db-4ece-a7f5-7f4ce35e4591 service placement] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:00.342884 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:00.400795 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287d764-e562-46a4-837c-6f196568bbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:00.401471 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287d764-e562-46a4-837c-6f196568bbad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:00.401763 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287d764-e562-46a4-837c-6f196568bbad admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:00.402017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287d764-e562-46a4-837c-6f196568bbad admin admin] PATH_INFO: `/v3/projects/c113bdd75f254073abcfbd109dc7d187` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:00.403333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8287d764-e562-46a4-837c-6f196568bbad admin admin] RBAC: Authorizing `identity:delete_project(project_id=c113bdd75f254073abcfbd109dc7d187)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:00.416509 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8287d764-e562-46a4-837c-6f196568bbad admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:00.545258 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8287d764-e562-46a4-837c-6f196568bbad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c113bdd75f254073abcfbd109dc7d187', 'request_id': 'req-8287d764-e562-46a4-837c-6f196568bbad'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:00.555955 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8287d764-e562-46a4-837c-6f196568bbad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c113bdd75f254073abcfbd109dc7d187', 'request_id': 'req-8287d764-e562-46a4-837c-6f196568bbad'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:00.571483 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8287d764-e562-46a4-837c-6f196568bbad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c113bdd75f254073abcfbd109dc7d187', 'request_id': 'req-8287d764-e562-46a4-837c-6f196568bbad'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:00.584647 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8287d764-e562-46a4-837c-6f196568bbad admin admin] The token cache is being invalidate because project c113bdd75f254073abcfbd109dc7d187 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:33:00.584942 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8287d764-e562-46a4-837c-6f196568bbad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:00.585666 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8287d764-e562-46a4-837c-6f196568bbad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:00.591017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1138/2268] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:33:00 2022] DELETE /identity/v3/projects/c113bdd75f254073abcfbd109dc7d187 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:00.592747 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4253fb-8b9d-4e5c-8135-f07e1a964ae6 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:00.593200 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4253fb-8b9d-4e5c-8135-f07e1a964ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:00.593305 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4253fb-8b9d-4e5c-8135-f07e1a964ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:00.618044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-3d4253fb-8b9d-4e5c-8135-f07e1a964ae6 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:00.620045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-3d4253fb-8b9d-4e5c-8135-f07e1a964ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:00.631395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:00.763096 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1131/2269] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:00.764388 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5f558-a6c0-4682-a693-2fcedebfe83a None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:00.764682 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5f558-a6c0-4682-a693-2fcedebfe83a None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:00.764961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5f558-a6c0-4682-a693-2fcedebfe83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:00.792308 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-e1e5f558-a6c0-4682-a693-2fcedebfe83a None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:00.793912 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-e1e5f558-a6c0-4682-a693-2fcedebfe83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:00.811422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1139/2270] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:00.815484 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4253fb-8b9d-4e5c-8135-f07e1a964ae6 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:00.827962 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4253fb-8b9d-4e5c-8135-f07e1a964ae6 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:00.834517 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc383c4-8825-4fd5-8de5-9f04fe119ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:00.835596 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc383c4-8825-4fd5-8de5-9f04fe119ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:00.836696 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc383c4-8825-4fd5-8de5-9f04fe119ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:00.837237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc383c4-8825-4fd5-8de5-9f04fe119ce5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:00.838947 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc383c4-8825-4fd5-8de5-9f04fe119ce5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:33:00.839407 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc383c4-8825-4fd5-8de5-9f04fe119ce5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:00.841825 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc383c4-8825-4fd5-8de5-9f04fe119ce5 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:00.859849 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1132/2271] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:33:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:33:00.864225 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e5f558-a6c0-4682-a693-2fcedebfe83a None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:00.874822 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e5f558-a6c0-4682-a693-2fcedebfe83a None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:00.880862 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3b39b-aee5-4822-a523-72ed3ce24a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:00.881553 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3b39b-aee5-4822-a523-72ed3ce24a83 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:00.881915 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3b39b-aee5-4822-a523-72ed3ce24a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:00.882253 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3b39b-aee5-4822-a523-72ed3ce24a83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:00.882947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a3b39b-aee5-4822-a523-72ed3ce24a83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:00.885043 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a3b39b-aee5-4822-a523-72ed3ce24a83 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:00.928606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1140/2272] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:33:00 2022] POST /identity/v3/projects => generated 363 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:33:00.933735 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc383c4-8825-4fd5-8de5-9f04fe119ce5 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:00.945847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc383c4-8825-4fd5-8de5-9f04fe119ce5 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:01.017747 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b77d7-8726-4020-8c55-1e5b473deb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:01.018286 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b77d7-8726-4020-8c55-1e5b473deb6b admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:01.018570 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b77d7-8726-4020-8c55-1e5b473deb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:01.018849 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b77d7-8726-4020-8c55-1e5b473deb6b admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:01.019494 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b77d7-8726-4020-8c55-1e5b473deb6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:01.021557 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b77d7-8726-4020-8c55-1e5b473deb6b admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:01.047272 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1133/2273] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:33:00 2022] POST /identity/v3/users => generated 282 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:01.051804 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3b39b-aee5-4822-a523-72ed3ce24a83 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:01.064516 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3b39b-aee5-4822-a523-72ed3ce24a83 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:01.069626 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dddda4c-c618-4085-acc4-ce63e5c3f8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:01.070252 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddda4c-c618-4085-acc4-ce63e5c3f8de admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:01.070534 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddda4c-c618-4085-acc4-ce63e5c3f8de admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:01.070808 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddda4c-c618-4085-acc4-ce63e5c3f8de admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:01.071457 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dddda4c-c618-4085-acc4-ce63e5c3f8de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:01.073185 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dddda4c-c618-4085-acc4-ce63e5c3f8de admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:01.085303 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1141/2274] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:01.089645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b77d7-8726-4020-8c55-1e5b473deb6b admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:01.100926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b77d7-8726-4020-8c55-1e5b473deb6b admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:01.111200 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c0044f-814e-4464-b72f-756345ae2c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:01.112041 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0044f-814e-4464-b72f-756345ae2c03 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:01.112373 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0044f-814e-4464-b72f-756345ae2c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:01.112711 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0044f-814e-4464-b72f-756345ae2c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:01.113488 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c0044f-814e-4464-b72f-756345ae2c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:01.115341 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c0044f-814e-4464-b72f-756345ae2c03 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:01.134531 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1134/2275] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:01.136105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dddda4c-c618-4085-acc4-ce63e5c3f8de admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:01.146974 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dddda4c-c618-4085-acc4-ce63e5c3f8de admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:01.152006 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c2675-531f-4b4e-ac41-77ac36d38d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:01.152689 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c2675-531f-4b4e-ac41-77ac36d38d7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:01.152968 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c2675-531f-4b4e-ac41-77ac36d38d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:01.153247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c2675-531f-4b4e-ac41-77ac36d38d7b admin admin] PATH_INFO: `/v3/projects/e45fefd375ec44aa93c753054c3e2d8e/users/70d1d3433abb48c997e1cad14c456df8/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:01.170215 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c2675-531f-4b4e-ac41-77ac36d38d7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e45fefd375ec44aa93c753054c3e2d8e, user_id=70d1d3433abb48c997e1cad14c456df8, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:01.173043 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c2675-531f-4b4e-ac41-77ac36d38d7b admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:01.189169 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1142/2276] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:33:01 2022] PUT /identity/v3/projects/e45fefd375ec44aa93c753054c3e2d8e/users/70d1d3433abb48c997e1cad14c456df8/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:01.201294 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ee717-1a57-4930-b2e6-1759d8dfcb7e None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:01.201585 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ee717-1a57-4930-b2e6-1759d8dfcb7e None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:01.201861 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ee717-1a57-4930-b2e6-1759d8dfcb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:01.233820 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-a76ee717-1a57-4930-b2e6-1759d8dfcb7e None None] MFA Rules not processed for user `70d1d3433abb48c997e1cad14c456df8`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:01.235169 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-a76ee717-1a57-4930-b2e6-1759d8dfcb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:01.320752 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1135/2277] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:01.326966 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c2675-531f-4b4e-ac41-77ac36d38d7b admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:01.398017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c2675-531f-4b4e-ac41-77ac36d38d7b admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:01.403168 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced5767-42dd-4c69-afec-e3abbb19e1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:01.403805 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced5767-42dd-4c69-afec-e3abbb19e1dd admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:01.404131 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced5767-42dd-4c69-afec-e3abbb19e1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:01.404441 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced5767-42dd-4c69-afec-e3abbb19e1dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:01.405137 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced5767-42dd-4c69-afec-e3abbb19e1dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:01.407134 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced5767-42dd-4c69-afec-e3abbb19e1dd admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:01.412529 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:01.443130 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1143/2278] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:33:01 2022] POST /identity/v3/projects => generated 363 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:01.447296 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ee717-1a57-4930-b2e6-1759d8dfcb7e None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:01.469433 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ee717-1a57-4930-b2e6-1759d8dfcb7e None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:01.479078 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:01.550410 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f69ce-b46a-4449-8ab1-23a6a4e99a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:01.551527 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f69ce-b46a-4449-8ab1-23a6a4e99a59 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:01.551979 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f69ce-b46a-4449-8ab1-23a6a4e99a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:01.552371 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f69ce-b46a-4449-8ab1-23a6a4e99a59 admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:01.553118 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f69ce-b46a-4449-8ab1-23a6a4e99a59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:01.555251 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f69ce-b46a-4449-8ab1-23a6a4e99a59 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:01.583789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced5767-42dd-4c69-afec-e3abbb19e1dd admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:01.584894 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1136/2279] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:33:01 2022] POST /identity/v3/users => generated 288 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:01.595815 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced5767-42dd-4c69-afec-e3abbb19e1dd admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:01.603130 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d2d3d-032a-42e3-bb7e-6231db6172bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:01.603852 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d2d3d-032a-42e3-bb7e-6231db6172bb admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:01.604410 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d2d3d-032a-42e3-bb7e-6231db6172bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:01.604885 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d2d3d-032a-42e3-bb7e-6231db6172bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:01.605695 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d2d3d-032a-42e3-bb7e-6231db6172bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:01.607610 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d2d3d-032a-42e3-bb7e-6231db6172bb admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:01.622580 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1144/2280] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:01.623770 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f69ce-b46a-4449-8ab1-23a6a4e99a59 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:01.634124 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f69ce-b46a-4449-8ab1-23a6a4e99a59 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:01.639513 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8b6e0-3943-46b9-bf61-fc0483bf3362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:01.640329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8b6e0-3943-46b9-bf61-fc0483bf3362 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:01.640732 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8b6e0-3943-46b9-bf61-fc0483bf3362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:01.641119 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8b6e0-3943-46b9-bf61-fc0483bf3362 admin admin] PATH_INFO: `/v3/projects/282296217a2d4e9cbb85fe70ad9cac3e/users/8e5a404b458e4e29a1d84b2710da995c/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:01.660887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e8b6e0-3943-46b9-bf61-fc0483bf3362 admin admin] RBAC: Authorizing `identity:create_grant(project_id=282296217a2d4e9cbb85fe70ad9cac3e, user_id=8e5a404b458e4e29a1d84b2710da995c, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:01.663996 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e8b6e0-3943-46b9-bf61-fc0483bf3362 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:01.681826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1137/2281] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:33:01 2022] PUT /identity/v3/projects/282296217a2d4e9cbb85fe70ad9cac3e/users/8e5a404b458e4e29a1d84b2710da995c/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:01.685918 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d2d3d-032a-42e3-bb7e-6231db6172bb admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:01.699054 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d2d3d-032a-42e3-bb7e-6231db6172bb admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:01.767704 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe17853-4447-4dd0-882e-efd1b910bbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:01.768507 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe17853-4447-4dd0-882e-efd1b910bbec admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:01.768798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe17853-4447-4dd0-882e-efd1b910bbec admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:01.769074 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe17853-4447-4dd0-882e-efd1b910bbec admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:01.769736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe17853-4447-4dd0-882e-efd1b910bbec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:01.771515 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe17853-4447-4dd0-882e-efd1b910bbec admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:01.783968 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1145/2282] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:01.787327 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8b6e0-3943-46b9-bf61-fc0483bf3362 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:01.801333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8b6e0-3943-46b9-bf61-fc0483bf3362 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:01.806208 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd59386-e311-41a5-8150-21244155b123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:01.806893 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd59386-e311-41a5-8150-21244155b123 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:01.807168 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd59386-e311-41a5-8150-21244155b123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:01.807436 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd59386-e311-41a5-8150-21244155b123 admin admin] PATH_INFO: `/v3/projects/282296217a2d4e9cbb85fe70ad9cac3e/users/8e5a404b458e4e29a1d84b2710da995c/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:01.809359 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd59386-e311-41a5-8150-21244155b123 admin admin] RBAC: Authorizing `identity:create_grant(project_id=282296217a2d4e9cbb85fe70ad9cac3e, user_id=8e5a404b458e4e29a1d84b2710da995c, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:01.812404 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd59386-e311-41a5-8150-21244155b123 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:01.825766 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1138/2283] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:33:01 2022] PUT /identity/v3/projects/282296217a2d4e9cbb85fe70ad9cac3e/users/8e5a404b458e4e29a1d84b2710da995c/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:01.837484 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9f910-49d5-4dc1-9efe-0e9884c788d6 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:01.838119 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9f910-49d5-4dc1-9efe-0e9884c788d6 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:01.838381 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9f910-49d5-4dc1-9efe-0e9884c788d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:01.879711 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-2bc9f910-49d5-4dc1-9efe-0e9884c788d6 None None] MFA Rules not processed for user `8e5a404b458e4e29a1d84b2710da995c`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:01.881776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-2bc9f910-49d5-4dc1-9efe-0e9884c788d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:02.008217 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1146/2284] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:01 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:33:02.009263 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd59386-e311-41a5-8150-21244155b123 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:02.011302 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-3bd59386-e311-41a5-8150-21244155b123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:02.090817 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd59386-e311-41a5-8150-21244155b123 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:02.099036 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08621d75-1459-4191-bd36-19f8fc4e1c75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:02.100094 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08621d75-1459-4191-bd36-19f8fc4e1c75 service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:02.100444 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08621d75-1459-4191-bd36-19f8fc4e1c75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:02.100793 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08621d75-1459-4191-bd36-19f8fc4e1c75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:02.110597 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08621d75-1459-4191-bd36-19f8fc4e1c75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:02.112279 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08621d75-1459-4191-bd36-19f8fc4e1c75 service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:02.128066 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1139/2285] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:02 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:33:02.678216 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9f910-49d5-4dc1-9efe-0e9884c788d6 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:02.748792 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:02.756518 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9f910-49d5-4dc1-9efe-0e9884c788d6 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:02.763577 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f8d56-83ad-434e-86ae-bc3f75ab9ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:02.764263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f8d56-83ad-434e-86ae-bc3f75ab9ed0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:02.764549 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f8d56-83ad-434e-86ae-bc3f75ab9ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:02.764875 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f8d56-83ad-434e-86ae-bc3f75ab9ed0 admin admin] PATH_INFO: `/v3/users/70d1d3433abb48c997e1cad14c456df8` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:02.766157 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f8d56-83ad-434e-86ae-bc3f75ab9ed0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70d1d3433abb48c997e1cad14c456df8)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:02.768905 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f8d56-83ad-434e-86ae-bc3f75ab9ed0 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:02.823572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-784f8d56-83ad-434e-86ae-bc3f75ab9ed0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70d1d3433abb48c997e1cad14c456df8'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:02.824351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-784f8d56-83ad-434e-86ae-bc3f75ab9ed0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70d1d3433abb48c997e1cad14c456df8'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:02.833032 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-784f8d56-83ad-434e-86ae-bc3f75ab9ed0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70d1d3433abb48c997e1cad14c456df8'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:02.848622 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-784f8d56-83ad-434e-86ae-bc3f75ab9ed0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70d1d3433abb48c997e1cad14c456df8'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:02.857536 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-784f8d56-83ad-434e-86ae-bc3f75ab9ed0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70d1d3433abb48c997e1cad14c456df8'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:02.865427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1147/2286] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:33:02 2022] DELETE /identity/v3/users/70d1d3433abb48c997e1cad14c456df8 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:02.869891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348189b-babc-4ba4-bf8a-accf422ca4a8 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:02.870765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348189b-babc-4ba4-bf8a-accf422ca4a8 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:02.871407 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348189b-babc-4ba4-bf8a-accf422ca4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:02.897321 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-b348189b-babc-4ba4-bf8a-accf422ca4a8 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:02.898717 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-b348189b-babc-4ba4-bf8a-accf422ca4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:02.964275 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1140/2287] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:02.970988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f8d56-83ad-434e-86ae-bc3f75ab9ed0 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:02.972442 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-784f8d56-83ad-434e-86ae-bc3f75ab9ed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:03.031715 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f8d56-83ad-434e-86ae-bc3f75ab9ed0 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:03.035792 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681828b2-787b-4dff-a9bd-987fdf7c658e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:03.036228 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681828b2-787b-4dff-a9bd-987fdf7c658e service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:03.036505 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681828b2-787b-4dff-a9bd-987fdf7c658e service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:03.036776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681828b2-787b-4dff-a9bd-987fdf7c658e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:03.047206 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681828b2-787b-4dff-a9bd-987fdf7c658e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:03.048803 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681828b2-787b-4dff-a9bd-987fdf7c658e service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:03.061197 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1148/2288] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:33:03.423471 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348189b-babc-4ba4-bf8a-accf422ca4a8 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:03.426348 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-b348189b-babc-4ba4-bf8a-accf422ca4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:03.438446 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348189b-babc-4ba4-bf8a-accf422ca4a8 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:03.444010 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2e69e-c82c-47e4-b595-c95227485e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:03.444710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2e69e-c82c-47e4-b595-c95227485e6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:03.444985 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2e69e-c82c-47e4-b595-c95227485e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:03.445259 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2e69e-c82c-47e4-b595-c95227485e6b admin admin] PATH_INFO: `/v3/projects/e45fefd375ec44aa93c753054c3e2d8e` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:03.446405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e2e69e-c82c-47e4-b595-c95227485e6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e45fefd375ec44aa93c753054c3e2d8e)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:03.448377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e2e69e-c82c-47e4-b595-c95227485e6b admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:03.450100 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:03.524509 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d3e2e69e-c82c-47e4-b595-c95227485e6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e45fefd375ec44aa93c753054c3e2d8e', 'request_id': 'req-d3e2e69e-c82c-47e4-b595-c95227485e6b'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:03.531025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d3e2e69e-c82c-47e4-b595-c95227485e6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e45fefd375ec44aa93c753054c3e2d8e', 'request_id': 'req-d3e2e69e-c82c-47e4-b595-c95227485e6b'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:03.541689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d3e2e69e-c82c-47e4-b595-c95227485e6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e45fefd375ec44aa93c753054c3e2d8e', 'request_id': 'req-d3e2e69e-c82c-47e4-b595-c95227485e6b'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:03.554765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d3e2e69e-c82c-47e4-b595-c95227485e6b admin admin] The token cache is being invalidate because project e45fefd375ec44aa93c753054c3e2d8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:33:03.556006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d3e2e69e-c82c-47e4-b595-c95227485e6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:03.557712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d3e2e69e-c82c-47e4-b595-c95227485e6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:03.560203 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1141/2289] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:33:03 2022] DELETE /identity/v3/projects/e45fefd375ec44aa93c753054c3e2d8e => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:03.564395 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681828b2-787b-4dff-a9bd-987fdf7c658e service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:03.574401 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681828b2-787b-4dff-a9bd-987fdf7c658e service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:03.650776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c0cdd5-8f8a-44b4-b9b0-fe5d869cab2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:03.651332 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0cdd5-8f8a-44b4-b9b0-fe5d869cab2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:03.651714 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0cdd5-8f8a-44b4-b9b0-fe5d869cab2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:03.652120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0cdd5-8f8a-44b4-b9b0-fe5d869cab2a admin admin] PATH_INFO: `/v3/users/8e5a404b458e4e29a1d84b2710da995c` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:03.653624 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c0cdd5-8f8a-44b4-b9b0-fe5d869cab2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e5a404b458e4e29a1d84b2710da995c)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:03.655741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c0cdd5-8f8a-44b4-b9b0-fe5d869cab2a admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:03.680631 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:03.723953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c8c0cdd5-8f8a-44b4-b9b0-fe5d869cab2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e5a404b458e4e29a1d84b2710da995c'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:03.725065 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c8c0cdd5-8f8a-44b4-b9b0-fe5d869cab2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e5a404b458e4e29a1d84b2710da995c'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:03.737377 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c8c0cdd5-8f8a-44b4-b9b0-fe5d869cab2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e5a404b458e4e29a1d84b2710da995c'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:03.755289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c8c0cdd5-8f8a-44b4-b9b0-fe5d869cab2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e5a404b458e4e29a1d84b2710da995c'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:03.765305 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c8c0cdd5-8f8a-44b4-b9b0-fe5d869cab2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e5a404b458e4e29a1d84b2710da995c'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:03.776691 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1149/2290] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:33:03 2022] DELETE /identity/v3/users/8e5a404b458e4e29a1d84b2710da995c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:04.107398 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2e69e-c82c-47e4-b595-c95227485e6b admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:04.109178 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-d3e2e69e-c82c-47e4-b595-c95227485e6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:04.126267 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2e69e-c82c-47e4-b595-c95227485e6b admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:04.199556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e8c627-a577-41b2-9d54-d9d2ef6734dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:04.200451 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8c627-a577-41b2-9d54-d9d2ef6734dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:04.200735 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8c627-a577-41b2-9d54-d9d2ef6734dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:04.201045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8c627-a577-41b2-9d54-d9d2ef6734dc admin admin] PATH_INFO: `/v3/projects/282296217a2d4e9cbb85fe70ad9cac3e` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:04.202190 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e8c627-a577-41b2-9d54-d9d2ef6734dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=282296217a2d4e9cbb85fe70ad9cac3e)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:04.204275 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e8c627-a577-41b2-9d54-d9d2ef6734dc admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:04.260001 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d4e8c627-a577-41b2-9d54-d9d2ef6734dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '282296217a2d4e9cbb85fe70ad9cac3e', 'request_id': 'req-d4e8c627-a577-41b2-9d54-d9d2ef6734dc'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:04.268611 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d4e8c627-a577-41b2-9d54-d9d2ef6734dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '282296217a2d4e9cbb85fe70ad9cac3e', 'request_id': 'req-d4e8c627-a577-41b2-9d54-d9d2ef6734dc'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:04.277805 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d4e8c627-a577-41b2-9d54-d9d2ef6734dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '282296217a2d4e9cbb85fe70ad9cac3e', 'request_id': 'req-d4e8c627-a577-41b2-9d54-d9d2ef6734dc'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:04.290081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d4e8c627-a577-41b2-9d54-d9d2ef6734dc admin admin] The token cache is being invalidate because project 282296217a2d4e9cbb85fe70ad9cac3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:33:04.290247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d4e8c627-a577-41b2-9d54-d9d2ef6734dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:04.290913 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d4e8c627-a577-41b2-9d54-d9d2ef6734dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:04.292470 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1142/2291] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:33:04 2022] DELETE /identity/v3/projects/282296217a2d4e9cbb85fe70ad9cac3e => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:04.297450 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad07c7-6d1b-480b-8c97-82d77c4e6f20 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:04.297901 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad07c7-6d1b-480b-8c97-82d77c4e6f20 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:04.298104 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad07c7-6d1b-480b-8c97-82d77c4e6f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:04.319731 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-5dad07c7-6d1b-480b-8c97-82d77c4e6f20 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:04.321016 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-5dad07c7-6d1b-480b-8c97-82d77c4e6f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:04.395111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1150/2292] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:04.403297 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bdf2e-d1c5-489a-bec3-32ec90521de5 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:04.403673 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bdf2e-d1c5-489a-bec3-32ec90521de5 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:04.404053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bdf2e-d1c5-489a-bec3-32ec90521de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:04.425333 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-df0bdf2e-d1c5-489a-bec3-32ec90521de5 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:04.426538 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-df0bdf2e-d1c5-489a-bec3-32ec90521de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:04.440480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1143/2293] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:04.441696 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad07c7-6d1b-480b-8c97-82d77c4e6f20 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:04.451881 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad07c7-6d1b-480b-8c97-82d77c4e6f20 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:04.456986 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569da6d-0aad-4d34-8f59-9ffe454eebf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:04.457487 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569da6d-0aad-4d34-8f59-9ffe454eebf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:04.458183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569da6d-0aad-4d34-8f59-9ffe454eebf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:04.458464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569da6d-0aad-4d34-8f59-9ffe454eebf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:04.459123 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569da6d-0aad-4d34-8f59-9ffe454eebf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:33:04.459428 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569da6d-0aad-4d34-8f59-9ffe454eebf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:04.461241 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569da6d-0aad-4d34-8f59-9ffe454eebf9 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:04.478158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1151/2294] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:33:04.479295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bdf2e-d1c5-489a-bec3-32ec90521de5 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:04.489035 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bdf2e-d1c5-489a-bec3-32ec90521de5 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:04.494438 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a78c4-97a0-49bc-8559-6cda1de01069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:04.495127 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a78c4-97a0-49bc-8559-6cda1de01069 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:04.495480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a78c4-97a0-49bc-8559-6cda1de01069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:04.495830 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a78c4-97a0-49bc-8559-6cda1de01069 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:04.496637 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368a78c4-97a0-49bc-8559-6cda1de01069 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:04.498696 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368a78c4-97a0-49bc-8559-6cda1de01069 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:04.510523 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:04.530067 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1144/2295] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:33:04 2022] POST /identity/v3/projects => generated 363 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:04.535006 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569da6d-0aad-4d34-8f59-9ffe454eebf9 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:04.544898 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569da6d-0aad-4d34-8f59-9ffe454eebf9 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:04.600485 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865826c0-2fa1-4714-ba4d-7750608176c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:04.601067 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865826c0-2fa1-4714-ba4d-7750608176c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:04.601339 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865826c0-2fa1-4714-ba4d-7750608176c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:04.601607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865826c0-2fa1-4714-ba4d-7750608176c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:04.602269 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865826c0-2fa1-4714-ba4d-7750608176c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:04.604226 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865826c0-2fa1-4714-ba4d-7750608176c1 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:04.626404 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1152/2296] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:33:04 2022] POST /identity/v3/users => generated 282 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:33:04.630857 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a78c4-97a0-49bc-8559-6cda1de01069 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:04.641423 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a78c4-97a0-49bc-8559-6cda1de01069 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:04.646147 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11acae-0170-4466-a5b2-1ad9a2fcf829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:04.646746 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11acae-0170-4466-a5b2-1ad9a2fcf829 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:04.647023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11acae-0170-4466-a5b2-1ad9a2fcf829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:04.647293 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11acae-0170-4466-a5b2-1ad9a2fcf829 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:04.647998 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e11acae-0170-4466-a5b2-1ad9a2fcf829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:04.649617 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e11acae-0170-4466-a5b2-1ad9a2fcf829 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:04.659744 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1145/2297] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:04.663898 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865826c0-2fa1-4714-ba4d-7750608176c1 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:04.673930 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865826c0-2fa1-4714-ba4d-7750608176c1 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:04.678511 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42faab69-a169-4073-b911-f25f51733d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:04.679124 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42faab69-a169-4073-b911-f25f51733d24 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:04.679398 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42faab69-a169-4073-b911-f25f51733d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:04.679685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42faab69-a169-4073-b911-f25f51733d24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:04.680408 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42faab69-a169-4073-b911-f25f51733d24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:04.681994 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42faab69-a169-4073-b911-f25f51733d24 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:04.692804 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1153/2298] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:04.696858 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11acae-0170-4466-a5b2-1ad9a2fcf829 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:04.707398 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11acae-0170-4466-a5b2-1ad9a2fcf829 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:04.712237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6cfc01-b81e-4193-8b5b-009f98842bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:04.712900 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6cfc01-b81e-4193-8b5b-009f98842bb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:04.713181 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6cfc01-b81e-4193-8b5b-009f98842bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:04.713454 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6cfc01-b81e-4193-8b5b-009f98842bb9 admin admin] PATH_INFO: `/v3/projects/0ae9d412b82a47298124edbb7de9500d/users/6fb9bbe185304e56897641254099afd2/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:04.730401 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6cfc01-b81e-4193-8b5b-009f98842bb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ae9d412b82a47298124edbb7de9500d, user_id=6fb9bbe185304e56897641254099afd2, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:04.733228 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6cfc01-b81e-4193-8b5b-009f98842bb9 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:04.747330 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1146/2299] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:33:04 2022] PUT /identity/v3/projects/0ae9d412b82a47298124edbb7de9500d/users/6fb9bbe185304e56897641254099afd2/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:04.758836 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136ad21-128a-4620-8c8c-664b9f06f32b None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:04.759126 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136ad21-128a-4620-8c8c-664b9f06f32b None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:04.759451 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136ad21-128a-4620-8c8c-664b9f06f32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:04.767235 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:04.790435 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-7136ad21-128a-4620-8c8c-664b9f06f32b None None] MFA Rules not processed for user `6fb9bbe185304e56897641254099afd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:04.791847 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-7136ad21-128a-4620-8c8c-664b9f06f32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:04.867194 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1154/2300] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:04.871940 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6cfc01-b81e-4193-8b5b-009f98842bb9 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:04.934811 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6cfc01-b81e-4193-8b5b-009f98842bb9 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:04.941676 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b350b-4fed-41a2-8c5c-48fba09daa78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:04.942331 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b350b-4fed-41a2-8c5c-48fba09daa78 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:04.942579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b350b-4fed-41a2-8c5c-48fba09daa78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:04.942847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b350b-4fed-41a2-8c5c-48fba09daa78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:04.943477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b350b-4fed-41a2-8c5c-48fba09daa78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:04.945741 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b350b-4fed-41a2-8c5c-48fba09daa78 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:04.975472 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1147/2301] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:33:04 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:04.979882 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136ad21-128a-4620-8c8c-664b9f06f32b None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:04.990948 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136ad21-128a-4620-8c8c-664b9f06f32b None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:05.060556 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28649780-27a0-46f3-a77a-4ff03d8e8c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:05.061212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28649780-27a0-46f3-a77a-4ff03d8e8c9a admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:05.061489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28649780-27a0-46f3-a77a-4ff03d8e8c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:05.061761 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28649780-27a0-46f3-a77a-4ff03d8e8c9a admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:05.062451 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28649780-27a0-46f3-a77a-4ff03d8e8c9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:05.064532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28649780-27a0-46f3-a77a-4ff03d8e8c9a admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:05.089365 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1155/2302] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:33:04 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:05.093477 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b350b-4fed-41a2-8c5c-48fba09daa78 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:05.104654 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b350b-4fed-41a2-8c5c-48fba09daa78 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:05.109641 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b471f0-a9ca-425b-8eeb-175e2dadf18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:05.110292 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b471f0-a9ca-425b-8eeb-175e2dadf18e admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:05.110529 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b471f0-a9ca-425b-8eeb-175e2dadf18e admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:05.110924 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b471f0-a9ca-425b-8eeb-175e2dadf18e admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:05.111491 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b471f0-a9ca-425b-8eeb-175e2dadf18e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:05.113212 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b471f0-a9ca-425b-8eeb-175e2dadf18e admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:05.128743 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1148/2303] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:05.129197 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28649780-27a0-46f3-a77a-4ff03d8e8c9a admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:05.141594 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28649780-27a0-46f3-a77a-4ff03d8e8c9a admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:05.147034 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9952c57-72cd-48bd-979e-7fb51248b7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:05.148009 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9952c57-72cd-48bd-979e-7fb51248b7c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:05.148289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9952c57-72cd-48bd-979e-7fb51248b7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:05.148559 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9952c57-72cd-48bd-979e-7fb51248b7c8 admin admin] PATH_INFO: `/v3/projects/f0e5a8db17d14aae98c61f5ff9391284/users/98925343bfa84fa797c64c2c1cb099aa/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:05.168637 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9952c57-72cd-48bd-979e-7fb51248b7c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0e5a8db17d14aae98c61f5ff9391284, user_id=98925343bfa84fa797c64c2c1cb099aa, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:05.172010 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9952c57-72cd-48bd-979e-7fb51248b7c8 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:05.194516 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1156/2304] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:33:05 2022] PUT /identity/v3/projects/f0e5a8db17d14aae98c61f5ff9391284/users/98925343bfa84fa797c64c2c1cb099aa/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:05.198914 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b471f0-a9ca-425b-8eeb-175e2dadf18e admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:05.214435 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b471f0-a9ca-425b-8eeb-175e2dadf18e admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:05.301405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d3d47-7d5b-4d88-bb9d-be7d2abfa48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:05.301924 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d3d47-7d5b-4d88-bb9d-be7d2abfa48d admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:05.302329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d3d47-7d5b-4d88-bb9d-be7d2abfa48d admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:05.302463 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d3d47-7d5b-4d88-bb9d-be7d2abfa48d admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:05.303197 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d3d47-7d5b-4d88-bb9d-be7d2abfa48d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:05.305501 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d3d47-7d5b-4d88-bb9d-be7d2abfa48d admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:05.319044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1149/2305] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:05.320675 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9952c57-72cd-48bd-979e-7fb51248b7c8 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:05.331256 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9952c57-72cd-48bd-979e-7fb51248b7c8 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:05.335851 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb71b2-40e3-4ef4-b174-16b88620a52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:05.336535 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb71b2-40e3-4ef4-b174-16b88620a52b admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:05.336809 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb71b2-40e3-4ef4-b174-16b88620a52b admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:05.337075 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb71b2-40e3-4ef4-b174-16b88620a52b admin admin] PATH_INFO: `/v3/projects/f0e5a8db17d14aae98c61f5ff9391284/users/98925343bfa84fa797c64c2c1cb099aa/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:05.338938 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcb71b2-40e3-4ef4-b174-16b88620a52b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0e5a8db17d14aae98c61f5ff9391284, user_id=98925343bfa84fa797c64c2c1cb099aa, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:05.341583 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcb71b2-40e3-4ef4-b174-16b88620a52b admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:05.355788 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1157/2306] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:33:05 2022] PUT /identity/v3/projects/f0e5a8db17d14aae98c61f5ff9391284/users/98925343bfa84fa797c64c2c1cb099aa/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:05.367101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42734c-8f9a-4ad9-911b-9fc08fac12da None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:05.367408 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42734c-8f9a-4ad9-911b-9fc08fac12da None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:05.368214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42734c-8f9a-4ad9-911b-9fc08fac12da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:05.397328 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-5d42734c-8f9a-4ad9-911b-9fc08fac12da None None] MFA Rules not processed for user `98925343bfa84fa797c64c2c1cb099aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:05.398532 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-5d42734c-8f9a-4ad9-911b-9fc08fac12da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:05.411907 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:05.478217 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1150/2307] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:05 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:05.491309 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb71b2-40e3-4ef4-b174-16b88620a52b admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:05.493250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-bbcb71b2-40e3-4ef4-b174-16b88620a52b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:05.539598 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:05.552568 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb71b2-40e3-4ef4-b174-16b88620a52b admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:05.556460 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852aa2b-44d1-4aa6-a7bb-51b3c57785f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:05.557290 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852aa2b-44d1-4aa6-a7bb-51b3c57785f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:05.557501 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852aa2b-44d1-4aa6-a7bb-51b3c57785f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:05.557724 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852aa2b-44d1-4aa6-a7bb-51b3c57785f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:05.567468 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852aa2b-44d1-4aa6-a7bb-51b3c57785f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:05.568938 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852aa2b-44d1-4aa6-a7bb-51b3c57785f0 service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:05.578540 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1158/2308] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:05 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:33:05.660421 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d42734c-8f9a-4ad9-911b-9fc08fac12da None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:05.671240 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d42734c-8f9a-4ad9-911b-9fc08fac12da None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:05.677447 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b7169-9f1d-49df-90b7-8b022e99ba20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:05.678152 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b7169-9f1d-49df-90b7-8b022e99ba20 service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:05.678486 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b7169-9f1d-49df-90b7-8b022e99ba20 service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:05.678830 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b7169-9f1d-49df-90b7-8b022e99ba20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:05.739817 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b7169-9f1d-49df-90b7-8b022e99ba20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:05.741418 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b7169-9f1d-49df-90b7-8b022e99ba20 service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:05.750883 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1151/2309] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:33:08.551832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852aa2b-44d1-4aa6-a7bb-51b3c57785f0 service cinder] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:08.554536 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-d852aa2b-44d1-4aa6-a7bb-51b3c57785f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:08.652215 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852aa2b-44d1-4aa6-a7bb-51b3c57785f0 service cinder] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:08.659851 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb14723e-bf35-4d19-95a4-6e31ce449391 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:08.660696 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb14723e-bf35-4d19-95a4-6e31ce449391 service nova] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:08.660979 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb14723e-bf35-4d19-95a4-6e31ce449391 service nova] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:08.661252 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb14723e-bf35-4d19-95a4-6e31ce449391 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:08.663229 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-fb14723e-bf35-4d19-95a4-6e31ce449391 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:08.688353 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb14723e-bf35-4d19-95a4-6e31ce449391 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:08.691993 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb14723e-bf35-4d19-95a4-6e31ce449391 service nova] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:08.707447 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1159/2310] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:33:14.824100 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374de57-ada9-423a-a2ba-2e2cadcd62c3 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:14.824622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374de57-ada9-423a-a2ba-2e2cadcd62c3 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:14.824622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374de57-ada9-423a-a2ba-2e2cadcd62c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:14.844087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-2374de57-ada9-423a-a2ba-2e2cadcd62c3 None None] MFA Rules not processed for user `d3538271ebfd41aabe9f3da85feb7bca`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:14.845250 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-2374de57-ada9-423a-a2ba-2e2cadcd62c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:14.904452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1152/2311] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Aug 23 04:33:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:14.916666 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb14723e-bf35-4d19-95a4-6e31ce449391 service nova] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:14.918099 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-fb14723e-bf35-4d19-95a4-6e31ce449391 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:14.987501 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb14723e-bf35-4d19-95a4-6e31ce449391 service nova] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:14.992967 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e9406-400a-4694-be88-7e503fe5f986 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f1418fd211a448da1a50175dce13b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:14.993704 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e9406-400a-4694-be88-7e503fe5f986 service swift] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:14.993986 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e9406-400a-4694-be88-7e503fe5f986 service swift] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:14.994319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e9406-400a-4694-be88-7e503fe5f986 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:15.003347 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e9406-400a-4694-be88-7e503fe5f986 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:15.004977 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e9406-400a-4694-be88-7e503fe5f986 service swift] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:15.013751 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1160/2312] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Aug 23 04:33:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 04:33:15.315803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2374de57-ada9-423a-a2ba-2e2cadcd62c3 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:15.412182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:15.421573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2374de57-ada9-423a-a2ba-2e2cadcd62c3 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:15.465936 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d00976b-6e6f-4fcd-980d-ff808dddcc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:15.466675 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d00976b-6e6f-4fcd-980d-ff808dddcc4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:15.466972 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d00976b-6e6f-4fcd-980d-ff808dddcc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:15.467228 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d00976b-6e6f-4fcd-980d-ff808dddcc4e admin admin] PATH_INFO: `/v3/users/6fb9bbe185304e56897641254099afd2` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:15.471653 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d00976b-6e6f-4fcd-980d-ff808dddcc4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fb9bbe185304e56897641254099afd2)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:15.473345 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d00976b-6e6f-4fcd-980d-ff808dddcc4e admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:15.573184 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-6d00976b-6e6f-4fcd-980d-ff808dddcc4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fb9bbe185304e56897641254099afd2'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:15.589396 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-6d00976b-6e6f-4fcd-980d-ff808dddcc4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fb9bbe185304e56897641254099afd2'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:15.597553 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e9406-400a-4694-be88-7e503fe5f986 service swift] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:15.617964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-6d00976b-6e6f-4fcd-980d-ff808dddcc4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fb9bbe185304e56897641254099afd2'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:15.618796 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-6d00976b-6e6f-4fcd-980d-ff808dddcc4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fb9bbe185304e56897641254099afd2'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:15.637541 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-6d00976b-6e6f-4fcd-980d-ff808dddcc4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fb9bbe185304e56897641254099afd2'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:15.642259 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e9406-400a-4694-be88-7e503fe5f986 service swift] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:15.651651 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1153/2313] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:33:15 2022] DELETE /identity/v3/users/6fb9bbe185304e56897641254099afd2 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:15.656485 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0d716-c43c-495c-b4aa-e37f4d2fed39 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:15.656781 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0d716-c43c-495c-b4aa-e37f4d2fed39 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:15.657112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0d716-c43c-495c-b4aa-e37f4d2fed39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:15.688531 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5b8a6-ef0c-40db-b50e-f9cc99d2cbbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:15.692343 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5b8a6-ef0c-40db-b50e-f9cc99d2cbbc service nova] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:15.692839 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5b8a6-ef0c-40db-b50e-f9cc99d2cbbc service nova] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:15.694376 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5b8a6-ef0c-40db-b50e-f9cc99d2cbbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:15.696599 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-65e0d716-c43c-495c-b4aa-e37f4d2fed39 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:15.697849 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-65e0d716-c43c-495c-b4aa-e37f4d2fed39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:15.703800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-c7a5b8a6-ef0c-40db-b50e-f9cc99d2cbbc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:15.766819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a5b8a6-ef0c-40db-b50e-f9cc99d2cbbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:15.768563 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a5b8a6-ef0c-40db-b50e-f9cc99d2cbbc service nova] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:15.819648 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1161/2314] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:33:15.843098 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1154/2315] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:33:15.854179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5b8a6-ef0c-40db-b50e-f9cc99d2cbbc service nova] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:15.855621 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-c7a5b8a6-ef0c-40db-b50e-f9cc99d2cbbc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:15.970443 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:15.975807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5b8a6-ef0c-40db-b50e-f9cc99d2cbbc service nova] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:16.029570 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c671b7-5018-4683-bddc-ad44c382599c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:16.030196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c671b7-5018-4683-bddc-ad44c382599c service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:16.030470 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c671b7-5018-4683-bddc-ad44c382599c service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:16.030742 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c671b7-5018-4683-bddc-ad44c382599c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:16.041096 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c671b7-5018-4683-bddc-ad44c382599c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:16.042610 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c671b7-5018-4683-bddc-ad44c382599c service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:16.064783 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1162/2316] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:33:16.557854 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e0d716-c43c-495c-b4aa-e37f4d2fed39 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:16.559633 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-65e0d716-c43c-495c-b4aa-e37f4d2fed39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:16.575671 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e0d716-c43c-495c-b4aa-e37f4d2fed39 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:16.587795 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85567e-58df-45a8-82ec-96a50ca3cb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:16.589508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85567e-58df-45a8-82ec-96a50ca3cb2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:16.589900 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85567e-58df-45a8-82ec-96a50ca3cb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:16.590265 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85567e-58df-45a8-82ec-96a50ca3cb2e admin admin] PATH_INFO: `/v3/projects/0ae9d412b82a47298124edbb7de9500d` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:16.591703 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de85567e-58df-45a8-82ec-96a50ca3cb2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ae9d412b82a47298124edbb7de9500d)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:16.593844 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de85567e-58df-45a8-82ec-96a50ca3cb2e admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:16.687944 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-de85567e-58df-45a8-82ec-96a50ca3cb2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ae9d412b82a47298124edbb7de9500d', 'request_id': 'req-de85567e-58df-45a8-82ec-96a50ca3cb2e'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:16.699064 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-de85567e-58df-45a8-82ec-96a50ca3cb2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ae9d412b82a47298124edbb7de9500d', 'request_id': 'req-de85567e-58df-45a8-82ec-96a50ca3cb2e'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:16.711328 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-de85567e-58df-45a8-82ec-96a50ca3cb2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ae9d412b82a47298124edbb7de9500d', 'request_id': 'req-de85567e-58df-45a8-82ec-96a50ca3cb2e'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:16.721039 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:16.728802 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-de85567e-58df-45a8-82ec-96a50ca3cb2e admin admin] The token cache is being invalidate because project 0ae9d412b82a47298124edbb7de9500d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:33:16.729733 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-de85567e-58df-45a8-82ec-96a50ca3cb2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:16.730738 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-de85567e-58df-45a8-82ec-96a50ca3cb2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:16.733542 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1155/2317] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:33:16 2022] DELETE /identity/v3/projects/0ae9d412b82a47298124edbb7de9500d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:16.738374 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c671b7-5018-4683-bddc-ad44c382599c service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:16.756751 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c671b7-5018-4683-bddc-ad44c382599c service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:16.892119 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7f379-ee8f-49e1-8c22-ff82f0d4c85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:16.893834 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7f379-ee8f-49e1-8c22-ff82f0d4c85e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:16.894346 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7f379-ee8f-49e1-8c22-ff82f0d4c85e admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:16.894791 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7f379-ee8f-49e1-8c22-ff82f0d4c85e admin admin] PATH_INFO: `/v3/users/98925343bfa84fa797c64c2c1cb099aa` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:16.896460 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7f379-ee8f-49e1-8c22-ff82f0d4c85e admin admin] RBAC: Authorizing `identity:delete_user(user_id=98925343bfa84fa797c64c2c1cb099aa)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:16.908878 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7f379-ee8f-49e1-8c22-ff82f0d4c85e admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:16.997789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c8c7f379-ee8f-49e1-8c22-ff82f0d4c85e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98925343bfa84fa797c64c2c1cb099aa'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:16.999649 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c8c7f379-ee8f-49e1-8c22-ff82f0d4c85e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98925343bfa84fa797c64c2c1cb099aa'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:17.017534 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c8c7f379-ee8f-49e1-8c22-ff82f0d4c85e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98925343bfa84fa797c64c2c1cb099aa'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:17.031626 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c8c7f379-ee8f-49e1-8c22-ff82f0d4c85e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98925343bfa84fa797c64c2c1cb099aa'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:17.050167 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c8c7f379-ee8f-49e1-8c22-ff82f0d4c85e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98925343bfa84fa797c64c2c1cb099aa'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:17.057981 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1163/2318] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:33:16 2022] DELETE /identity/v3/users/98925343bfa84fa797c64c2c1cb099aa => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:17.656588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85567e-58df-45a8-82ec-96a50ca3cb2e admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:17.658227 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-de85567e-58df-45a8-82ec-96a50ca3cb2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:17.684713 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85567e-58df-45a8-82ec-96a50ca3cb2e admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:17.781077 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6775a-aec9-433c-a681-77cea3ce58db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:17.782073 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6775a-aec9-433c-a681-77cea3ce58db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:17.782522 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6775a-aec9-433c-a681-77cea3ce58db admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:17.783006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6775a-aec9-433c-a681-77cea3ce58db admin admin] PATH_INFO: `/v3/projects/f0e5a8db17d14aae98c61f5ff9391284` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:17.784395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f6775a-aec9-433c-a681-77cea3ce58db admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0e5a8db17d14aae98c61f5ff9391284)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:17.786783 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f6775a-aec9-433c-a681-77cea3ce58db admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:17.842974 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-26f6775a-aec9-433c-a681-77cea3ce58db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0e5a8db17d14aae98c61f5ff9391284', 'request_id': 'req-26f6775a-aec9-433c-a681-77cea3ce58db'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:17.851309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-26f6775a-aec9-433c-a681-77cea3ce58db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0e5a8db17d14aae98c61f5ff9391284', 'request_id': 'req-26f6775a-aec9-433c-a681-77cea3ce58db'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:17.861801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-26f6775a-aec9-433c-a681-77cea3ce58db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0e5a8db17d14aae98c61f5ff9391284', 'request_id': 'req-26f6775a-aec9-433c-a681-77cea3ce58db'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:17.873294 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-26f6775a-aec9-433c-a681-77cea3ce58db admin admin] The token cache is being invalidate because project f0e5a8db17d14aae98c61f5ff9391284 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:33:17.873585 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-26f6775a-aec9-433c-a681-77cea3ce58db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:17.874457 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-26f6775a-aec9-433c-a681-77cea3ce58db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:17.876534 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1156/2319] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:33:17 2022] DELETE /identity/v3/projects/f0e5a8db17d14aae98c61f5ff9391284 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:17.881707 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7cc4b-1aeb-4ce4-9e2c-6d42f20cdcaf None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:17.882012 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7cc4b-1aeb-4ce4-9e2c-6d42f20cdcaf None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:17.882284 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7cc4b-1aeb-4ce4-9e2c-6d42f20cdcaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:17.912513 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-4ab7cc4b-1aeb-4ce4-9e2c-6d42f20cdcaf None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:17.913986 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-4ab7cc4b-1aeb-4ce4-9e2c-6d42f20cdcaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:17.941987 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:17.992720 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1164/2320] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:18.000818 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc05ad1-423f-4213-bf28-ead8c8061c6d None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:18.001218 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc05ad1-423f-4213-bf28-ead8c8061c6d None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:18.001568 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc05ad1-423f-4213-bf28-ead8c8061c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:18.023396 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-0bc05ad1-423f-4213-bf28-ead8c8061c6d None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:18.024942 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-0bc05ad1-423f-4213-bf28-ead8c8061c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:18.033399 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1157/2321] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:18.038013 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7cc4b-1aeb-4ce4-9e2c-6d42f20cdcaf None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:18.050177 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7cc4b-1aeb-4ce4-9e2c-6d42f20cdcaf None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:18.055641 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e95a622-bf8e-4efa-a2e6-26fc4010dcbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:18.056304 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95a622-bf8e-4efa-a2e6-26fc4010dcbd admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:18.056584 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95a622-bf8e-4efa-a2e6-26fc4010dcbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:18.056860 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95a622-bf8e-4efa-a2e6-26fc4010dcbd admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:18.057550 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e95a622-bf8e-4efa-a2e6-26fc4010dcbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:33:18.057856 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e95a622-bf8e-4efa-a2e6-26fc4010dcbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:18.059553 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e95a622-bf8e-4efa-a2e6-26fc4010dcbd admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:18.092554 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1165/2322] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:33:18.093726 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc05ad1-423f-4213-bf28-ead8c8061c6d None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:18.104964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc05ad1-423f-4213-bf28-ead8c8061c6d None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:18.110160 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0f3c8-ee23-49ea-98b0-f7ea0d30ba3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:18.111029 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0f3c8-ee23-49ea-98b0-f7ea0d30ba3d admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:18.111570 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0f3c8-ee23-49ea-98b0-f7ea0d30ba3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:18.111990 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0f3c8-ee23-49ea-98b0-f7ea0d30ba3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:18.112710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f0f3c8-ee23-49ea-98b0-f7ea0d30ba3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:18.114749 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f0f3c8-ee23-49ea-98b0-f7ea0d30ba3d admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:18.147953 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1158/2323] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:33:18 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:33:18.157854 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e95a622-bf8e-4efa-a2e6-26fc4010dcbd admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:18.169537 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e95a622-bf8e-4efa-a2e6-26fc4010dcbd admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:18.260017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddc3ff-4622-4e38-8252-e74f6ffc4152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:18.261324 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddc3ff-4622-4e38-8252-e74f6ffc4152 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:18.261995 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddc3ff-4622-4e38-8252-e74f6ffc4152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:18.262450 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddc3ff-4622-4e38-8252-e74f6ffc4152 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:18.263511 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ddc3ff-4622-4e38-8252-e74f6ffc4152 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:18.267027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ddc3ff-4622-4e38-8252-e74f6ffc4152 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:18.300182 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1166/2324] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:33:18 2022] POST /identity/v3/users => generated 285 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:18.305739 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0f3c8-ee23-49ea-98b0-f7ea0d30ba3d admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:18.323104 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0f3c8-ee23-49ea-98b0-f7ea0d30ba3d admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:18.330446 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98d2dc-bf3c-4cee-8eda-16e816ec609f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:18.331566 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98d2dc-bf3c-4cee-8eda-16e816ec609f admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:18.332053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98d2dc-bf3c-4cee-8eda-16e816ec609f admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:18.332515 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98d2dc-bf3c-4cee-8eda-16e816ec609f admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:18.333498 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f98d2dc-bf3c-4cee-8eda-16e816ec609f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:18.335123 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f98d2dc-bf3c-4cee-8eda-16e816ec609f admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:18.348232 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1159/2325] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:18.352719 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddc3ff-4622-4e38-8252-e74f6ffc4152 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:18.364477 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddc3ff-4622-4e38-8252-e74f6ffc4152 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:18.370654 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91898544-5f89-4ba7-8469-aade35a7dfd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:18.371282 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91898544-5f89-4ba7-8469-aade35a7dfd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:18.371558 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91898544-5f89-4ba7-8469-aade35a7dfd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:18.371834 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91898544-5f89-4ba7-8469-aade35a7dfd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:18.372521 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91898544-5f89-4ba7-8469-aade35a7dfd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:18.374149 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91898544-5f89-4ba7-8469-aade35a7dfd5 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:18.388343 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1167/2326] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:18.392529 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98d2dc-bf3c-4cee-8eda-16e816ec609f admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:18.403670 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98d2dc-bf3c-4cee-8eda-16e816ec609f admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:18.409658 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4214936-c2f4-45f4-8d9c-6173287c8a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:18.410559 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4214936-c2f4-45f4-8d9c-6173287c8a72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:18.411053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4214936-c2f4-45f4-8d9c-6173287c8a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:18.411508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4214936-c2f4-45f4-8d9c-6173287c8a72 admin admin] PATH_INFO: `/v3/projects/6c70c8dc8d364af8a085d874a955f9a8/users/5aff8ddee9074f63bd328f0091d355e8/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:18.436576 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4214936-c2f4-45f4-8d9c-6173287c8a72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c70c8dc8d364af8a085d874a955f9a8, user_id=5aff8ddee9074f63bd328f0091d355e8, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:18.442006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4214936-c2f4-45f4-8d9c-6173287c8a72 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:18.463988 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1160/2327] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:33:18 2022] PUT /identity/v3/projects/6c70c8dc8d364af8a085d874a955f9a8/users/5aff8ddee9074f63bd328f0091d355e8/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:18.477438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e938be3-1648-469f-9610-548ee6cf4715 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:18.478002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e938be3-1648-469f-9610-548ee6cf4715 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:18.478293 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e938be3-1648-469f-9610-548ee6cf4715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:18.528299 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-3e938be3-1648-469f-9610-548ee6cf4715 None None] MFA Rules not processed for user `5aff8ddee9074f63bd328f0091d355e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:18.530113 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-3e938be3-1648-469f-9610-548ee6cf4715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:18.640889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1168/2328] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:33:18.645247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4214936-c2f4-45f4-8d9c-6173287c8a72 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:18.677286 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:18.730306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4214936-c2f4-45f4-8d9c-6173287c8a72 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:18.735198 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26f001-1998-4487-99f3-a513555e4e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:18.735835 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26f001-1998-4487-99f3-a513555e4e61 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:18.736223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26f001-1998-4487-99f3-a513555e4e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:18.737176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26f001-1998-4487-99f3-a513555e4e61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:18.737823 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac26f001-1998-4487-99f3-a513555e4e61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:18.739764 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac26f001-1998-4487-99f3-a513555e4e61 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:18.780495 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1161/2329] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:33:18 2022] POST /identity/v3/projects => generated 369 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:18.786300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e938be3-1648-469f-9610-548ee6cf4715 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:18.800414 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e938be3-1648-469f-9610-548ee6cf4715 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:18.826465 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:18.878400 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024220a-e22b-440d-b6c9-0afa6c39cbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:18.879238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024220a-e22b-440d-b6c9-0afa6c39cbc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:18.879971 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024220a-e22b-440d-b6c9-0afa6c39cbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:18.880293 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024220a-e22b-440d-b6c9-0afa6c39cbc6 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:18.880921 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7024220a-e22b-440d-b6c9-0afa6c39cbc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:18.883207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7024220a-e22b-440d-b6c9-0afa6c39cbc6 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:18.910596 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1169/2330] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:33:18 2022] POST /identity/v3/users => generated 291 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:33:18.914746 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26f001-1998-4487-99f3-a513555e4e61 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:18.926142 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26f001-1998-4487-99f3-a513555e4e61 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:18.931323 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12440612-9e2c-470c-84d8-4b66244248f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:18.931988 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12440612-9e2c-470c-84d8-4b66244248f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:18.932306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12440612-9e2c-470c-84d8-4b66244248f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:18.932575 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12440612-9e2c-470c-84d8-4b66244248f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:18.933242 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12440612-9e2c-470c-84d8-4b66244248f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:18.934856 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12440612-9e2c-470c-84d8-4b66244248f6 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:18.951855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1162/2331] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:18.955560 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024220a-e22b-440d-b6c9-0afa6c39cbc6 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:18.966626 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024220a-e22b-440d-b6c9-0afa6c39cbc6 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:18.971930 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab25e6-bd3a-403a-a336-a473a9af494b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:18.972692 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab25e6-bd3a-403a-a336-a473a9af494b admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:18.972975 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab25e6-bd3a-403a-a336-a473a9af494b admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:18.973254 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab25e6-bd3a-403a-a336-a473a9af494b admin admin] PATH_INFO: `/v3/projects/87c54ea0a0304b7987e55dcf7070eeb0/users/add910366bdd4ef9913a23e36dc28721/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:18.994052 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab25e6-bd3a-403a-a336-a473a9af494b admin admin] RBAC: Authorizing `identity:create_grant(project_id=87c54ea0a0304b7987e55dcf7070eeb0, user_id=add910366bdd4ef9913a23e36dc28721, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:18.997294 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab25e6-bd3a-403a-a336-a473a9af494b admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:19.013165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1170/2332] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:33:18 2022] PUT /identity/v3/projects/87c54ea0a0304b7987e55dcf7070eeb0/users/add910366bdd4ef9913a23e36dc28721/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:19.017406 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12440612-9e2c-470c-84d8-4b66244248f6 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:19.027661 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12440612-9e2c-470c-84d8-4b66244248f6 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:19.101412 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f3d53-7fa8-4af3-b4b9-4b67ce717dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:19.101938 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f3d53-7fa8-4af3-b4b9-4b67ce717dbb admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:19.102225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f3d53-7fa8-4af3-b4b9-4b67ce717dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:19.102500 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f3d53-7fa8-4af3-b4b9-4b67ce717dbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:19.103142 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f3d53-7fa8-4af3-b4b9-4b67ce717dbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:19.104815 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f3d53-7fa8-4af3-b4b9-4b67ce717dbb admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:19.115666 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1163/2333] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:19.119816 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab25e6-bd3a-403a-a336-a473a9af494b admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:19.130681 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab25e6-bd3a-403a-a336-a473a9af494b admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:19.135518 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19900b-a906-4041-9851-39fde13e2004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:19.136219 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19900b-a906-4041-9851-39fde13e2004 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:19.136493 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19900b-a906-4041-9851-39fde13e2004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:19.136768 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19900b-a906-4041-9851-39fde13e2004 admin admin] PATH_INFO: `/v3/projects/87c54ea0a0304b7987e55dcf7070eeb0/users/add910366bdd4ef9913a23e36dc28721/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:19.138679 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef19900b-a906-4041-9851-39fde13e2004 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87c54ea0a0304b7987e55dcf7070eeb0, user_id=add910366bdd4ef9913a23e36dc28721, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:19.141358 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef19900b-a906-4041-9851-39fde13e2004 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:19.152344 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1171/2334] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:33:19 2022] PUT /identity/v3/projects/87c54ea0a0304b7987e55dcf7070eeb0/users/add910366bdd4ef9913a23e36dc28721/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:33:19.164235 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb76dc4-b005-46cd-b11b-ae6f24c3fea8 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:19.164525 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb76dc4-b005-46cd-b11b-ae6f24c3fea8 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:19.164804 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb76dc4-b005-46cd-b11b-ae6f24c3fea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:19.196838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-1cb76dc4-b005-46cd-b11b-ae6f24c3fea8 None None] MFA Rules not processed for user `add910366bdd4ef9913a23e36dc28721`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:19.198082 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-1cb76dc4-b005-46cd-b11b-ae6f24c3fea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:19.285968 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1164/2335] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:19 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:19.294733 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19900b-a906-4041-9851-39fde13e2004 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:19.296615 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-ef19900b-a906-4041-9851-39fde13e2004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:19.363742 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19900b-a906-4041-9851-39fde13e2004 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:19.367962 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b9a4e-623c-4391-a3c8-ab7146a8463a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:19.368657 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b9a4e-623c-4391-a3c8-ab7146a8463a service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:19.368934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b9a4e-623c-4391-a3c8-ab7146a8463a service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:19.369211 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b9a4e-623c-4391-a3c8-ab7146a8463a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:19.379270 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b9a4e-623c-4391-a3c8-ab7146a8463a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:19.380782 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b9a4e-623c-4391-a3c8-ab7146a8463a service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:19.390659 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1172/2336] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:19 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:33:19.445690 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb76dc4-b005-46cd-b11b-ae6f24c3fea8 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:19.447144 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-1cb76dc4-b005-46cd-b11b-ae6f24c3fea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:19.465383 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb76dc4-b005-46cd-b11b-ae6f24c3fea8 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:19.469735 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44491e8-7368-4931-a07f-c99485781dc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:19.470273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44491e8-7368-4931-a07f-c99485781dc4 service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:19.470757 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44491e8-7368-4931-a07f-c99485781dc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:19.471107 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44491e8-7368-4931-a07f-c99485781dc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:19.494138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:19.550556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44491e8-7368-4931-a07f-c99485781dc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:19.556762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44491e8-7368-4931-a07f-c99485781dc4 service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:19.577370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1165/2337] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:33:23.209812 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b9a4e-623c-4391-a3c8-ab7146a8463a service cinder] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:23.274581 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b9a4e-623c-4391-a3c8-ab7146a8463a service cinder] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:23.281016 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196964d-a438-469a-a051-207a5698cfb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:23.281842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196964d-a438-469a-a051-207a5698cfb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:23.282186 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196964d-a438-469a-a051-207a5698cfb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:23.282398 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196964d-a438-469a-a051-207a5698cfb7 admin admin] PATH_INFO: `/v3/users/5aff8ddee9074f63bd328f0091d355e8` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:23.283680 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196964d-a438-469a-a051-207a5698cfb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5aff8ddee9074f63bd328f0091d355e8)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:23.285814 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196964d-a438-469a-a051-207a5698cfb7 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:23.308045 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:23.339514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-0196964d-a438-469a-a051-207a5698cfb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5aff8ddee9074f63bd328f0091d355e8'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:23.340221 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-0196964d-a438-469a-a051-207a5698cfb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5aff8ddee9074f63bd328f0091d355e8'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:23.349711 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-0196964d-a438-469a-a051-207a5698cfb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5aff8ddee9074f63bd328f0091d355e8'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:23.366864 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-0196964d-a438-469a-a051-207a5698cfb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5aff8ddee9074f63bd328f0091d355e8'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:23.373872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-0196964d-a438-469a-a051-207a5698cfb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5aff8ddee9074f63bd328f0091d355e8'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:23.379922 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1173/2338] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:33:23 2022] DELETE /identity/v3/users/5aff8ddee9074f63bd328f0091d355e8 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:23.384635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33f7bc-e8e1-4f3e-b30b-cae68440ec89 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:23.385155 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33f7bc-e8e1-4f3e-b30b-cae68440ec89 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:23.385268 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33f7bc-e8e1-4f3e-b30b-cae68440ec89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:23.404056 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-4b33f7bc-e8e1-4f3e-b30b-cae68440ec89 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:23.405236 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-4b33f7bc-e8e1-4f3e-b30b-cae68440ec89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:23.471204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1166/2339] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:23.482203 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196964d-a438-469a-a051-207a5698cfb7 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:23.483772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-0196964d-a438-469a-a051-207a5698cfb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:23.553538 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196964d-a438-469a-a051-207a5698cfb7 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:23.557135 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4874efb8-aa51-49f2-9e8b-494670b80ac8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:23.557742 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874efb8-aa51-49f2-9e8b-494670b80ac8 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:23.558019 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874efb8-aa51-49f2-9e8b-494670b80ac8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:23.558270 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874efb8-aa51-49f2-9e8b-494670b80ac8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:23.568184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4874efb8-aa51-49f2-9e8b-494670b80ac8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:23.569631 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4874efb8-aa51-49f2-9e8b-494670b80ac8 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:23.579157 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1174/2340] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:33:23.859887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33f7bc-e8e1-4f3e-b30b-cae68440ec89 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:23.862022 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-4b33f7bc-e8e1-4f3e-b30b-cae68440ec89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:23.877706 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33f7bc-e8e1-4f3e-b30b-cae68440ec89 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:23.883306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a1386-e6dd-4b96-945c-c539db1e147c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:23.883969 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a1386-e6dd-4b96-945c-c539db1e147c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:23.884258 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a1386-e6dd-4b96-945c-c539db1e147c admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:23.884514 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a1386-e6dd-4b96-945c-c539db1e147c admin admin] PATH_INFO: `/v3/projects/6c70c8dc8d364af8a085d874a955f9a8` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:23.885634 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a1386-e6dd-4b96-945c-c539db1e147c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c70c8dc8d364af8a085d874a955f9a8)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:23.887571 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a1386-e6dd-4b96-945c-c539db1e147c admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:23.938065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-564a1386-e6dd-4b96-945c-c539db1e147c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c70c8dc8d364af8a085d874a955f9a8', 'request_id': 'req-564a1386-e6dd-4b96-945c-c539db1e147c'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:23.944363 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-564a1386-e6dd-4b96-945c-c539db1e147c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c70c8dc8d364af8a085d874a955f9a8', 'request_id': 'req-564a1386-e6dd-4b96-945c-c539db1e147c'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:23.953568 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-564a1386-e6dd-4b96-945c-c539db1e147c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c70c8dc8d364af8a085d874a955f9a8', 'request_id': 'req-564a1386-e6dd-4b96-945c-c539db1e147c'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:23.964470 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-564a1386-e6dd-4b96-945c-c539db1e147c admin admin] The token cache is being invalidate because project 6c70c8dc8d364af8a085d874a955f9a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:33:23.964754 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-564a1386-e6dd-4b96-945c-c539db1e147c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:23.965382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-564a1386-e6dd-4b96-945c-c539db1e147c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:23.966885 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1167/2341] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:33:23 2022] DELETE /identity/v3/projects/6c70c8dc8d364af8a085d874a955f9a8 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:23.970934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4874efb8-aa51-49f2-9e8b-494670b80ac8 service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:23.980915 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4874efb8-aa51-49f2-9e8b-494670b80ac8 service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:24.047866 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47054842-9c95-4da9-98ba-1491d3ae1563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:24.048811 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47054842-9c95-4da9-98ba-1491d3ae1563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:24.049197 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47054842-9c95-4da9-98ba-1491d3ae1563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:24.049568 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47054842-9c95-4da9-98ba-1491d3ae1563 admin admin] PATH_INFO: `/v3/users/add910366bdd4ef9913a23e36dc28721` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:24.051695 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47054842-9c95-4da9-98ba-1491d3ae1563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=add910366bdd4ef9913a23e36dc28721)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:24.054944 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47054842-9c95-4da9-98ba-1491d3ae1563 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:24.101508 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:24.107765 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-47054842-9c95-4da9-98ba-1491d3ae1563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'add910366bdd4ef9913a23e36dc28721'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:24.108874 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-47054842-9c95-4da9-98ba-1491d3ae1563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'add910366bdd4ef9913a23e36dc28721'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:24.119973 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-47054842-9c95-4da9-98ba-1491d3ae1563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'add910366bdd4ef9913a23e36dc28721'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:24.132490 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-47054842-9c95-4da9-98ba-1491d3ae1563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'add910366bdd4ef9913a23e36dc28721'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:24.139265 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-47054842-9c95-4da9-98ba-1491d3ae1563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'add910366bdd4ef9913a23e36dc28721'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:24.145406 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1175/2342] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:33:23 2022] DELETE /identity/v3/users/add910366bdd4ef9913a23e36dc28721 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:24.418131 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a1386-e6dd-4b96-945c-c539db1e147c admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:24.419699 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-564a1386-e6dd-4b96-945c-c539db1e147c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:24.434696 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a1386-e6dd-4b96-945c-c539db1e147c admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:24.512815 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:24.549016 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904708f5-cb03-49be-b704-44d1cbdd46a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:24.549670 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904708f5-cb03-49be-b704-44d1cbdd46a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:24.549952 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904708f5-cb03-49be-b704-44d1cbdd46a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:24.550225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904708f5-cb03-49be-b704-44d1cbdd46a8 admin admin] PATH_INFO: `/v3/projects/87c54ea0a0304b7987e55dcf7070eeb0` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:24.551414 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904708f5-cb03-49be-b704-44d1cbdd46a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87c54ea0a0304b7987e55dcf7070eeb0)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:24.554030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904708f5-cb03-49be-b704-44d1cbdd46a8 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:24.639028 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-904708f5-cb03-49be-b704-44d1cbdd46a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87c54ea0a0304b7987e55dcf7070eeb0', 'request_id': 'req-904708f5-cb03-49be-b704-44d1cbdd46a8'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:24.649269 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-904708f5-cb03-49be-b704-44d1cbdd46a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87c54ea0a0304b7987e55dcf7070eeb0', 'request_id': 'req-904708f5-cb03-49be-b704-44d1cbdd46a8'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:24.662490 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-904708f5-cb03-49be-b704-44d1cbdd46a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87c54ea0a0304b7987e55dcf7070eeb0', 'request_id': 'req-904708f5-cb03-49be-b704-44d1cbdd46a8'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:24.680427 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-904708f5-cb03-49be-b704-44d1cbdd46a8 admin admin] The token cache is being invalidate because project 87c54ea0a0304b7987e55dcf7070eeb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:33:24.680427 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-904708f5-cb03-49be-b704-44d1cbdd46a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:24.681534 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-904708f5-cb03-49be-b704-44d1cbdd46a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:24.683685 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1168/2343] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:33:24 2022] DELETE /identity/v3/projects/87c54ea0a0304b7987e55dcf7070eeb0 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:24.688970 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40eaaf3-cdea-4744-b8f9-c830efd78968 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:24.689629 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40eaaf3-cdea-4744-b8f9-c830efd78968 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:24.689913 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40eaaf3-cdea-4744-b8f9-c830efd78968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:24.714062 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-d40eaaf3-cdea-4744-b8f9-c830efd78968 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:24.715410 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-d40eaaf3-cdea-4744-b8f9-c830efd78968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:24.790084 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1176/2344] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:33:24.791819 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7b7d0-3b64-4c56-9127-6adad97fdbb6 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:24.792143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7b7d0-3b64-4c56-9127-6adad97fdbb6 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:24.792419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7b7d0-3b64-4c56-9127-6adad97fdbb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:24.828785 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-3bf7b7d0-3b64-4c56-9127-6adad97fdbb6 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:24.830244 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-3bf7b7d0-3b64-4c56-9127-6adad97fdbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:24.841476 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1169/2345] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:24.845856 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40eaaf3-cdea-4744-b8f9-c830efd78968 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:24.856658 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40eaaf3-cdea-4744-b8f9-c830efd78968 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:24.868338 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518360d1-db00-437f-820f-66fb5ed61e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:24.869089 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518360d1-db00-437f-820f-66fb5ed61e4f admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:24.870005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518360d1-db00-437f-820f-66fb5ed61e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:24.870416 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518360d1-db00-437f-820f-66fb5ed61e4f admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:24.871251 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518360d1-db00-437f-820f-66fb5ed61e4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:33:24.871695 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518360d1-db00-437f-820f-66fb5ed61e4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:24.873600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518360d1-db00-437f-820f-66fb5ed61e4f admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:24.889669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1177/2346] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:33:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:33:24.890923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7b7d0-3b64-4c56-9127-6adad97fdbb6 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:24.901076 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7b7d0-3b64-4c56-9127-6adad97fdbb6 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:24.906578 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5cd4f1-b38d-4128-a96a-dd21655b279e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:24.907189 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5cd4f1-b38d-4128-a96a-dd21655b279e admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:24.907482 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5cd4f1-b38d-4128-a96a-dd21655b279e admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:24.907719 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5cd4f1-b38d-4128-a96a-dd21655b279e admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:24.908381 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5cd4f1-b38d-4128-a96a-dd21655b279e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:24.910486 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5cd4f1-b38d-4128-a96a-dd21655b279e admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:24.942673 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1170/2347] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:33:24 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:24.946807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518360d1-db00-437f-820f-66fb5ed61e4f admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:24.958265 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518360d1-db00-437f-820f-66fb5ed61e4f admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:25.037206 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a87a48-13f7-4cfd-95b0-64ae12ec02bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:25.037995 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a87a48-13f7-4cfd-95b0-64ae12ec02bf admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:25.038384 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a87a48-13f7-4cfd-95b0-64ae12ec02bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:25.038759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a87a48-13f7-4cfd-95b0-64ae12ec02bf admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:25.039823 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a87a48-13f7-4cfd-95b0-64ae12ec02bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:25.042450 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a87a48-13f7-4cfd-95b0-64ae12ec02bf admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:25.072184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1178/2348] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:33:24 2022] POST /identity/v3/users => generated 294 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:25.077445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5cd4f1-b38d-4128-a96a-dd21655b279e admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:25.090554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5cd4f1-b38d-4128-a96a-dd21655b279e admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:25.095693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f27df6-7935-4361-a172-e50bb26718a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:25.096473 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f27df6-7935-4361-a172-e50bb26718a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:25.096756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f27df6-7935-4361-a172-e50bb26718a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:25.097021 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f27df6-7935-4361-a172-e50bb26718a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:25.097693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f27df6-7935-4361-a172-e50bb26718a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:25.099334 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f27df6-7935-4361-a172-e50bb26718a4 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:25.110694 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1171/2349] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:25.115586 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a87a48-13f7-4cfd-95b0-64ae12ec02bf admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:25.126304 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a87a48-13f7-4cfd-95b0-64ae12ec02bf admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:25.133765 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7da16-43f7-427f-9e0b-ec0462c091b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:25.134385 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7da16-43f7-427f-9e0b-ec0462c091b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:25.134660 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7da16-43f7-427f-9e0b-ec0462c091b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:25.134927 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7da16-43f7-427f-9e0b-ec0462c091b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:25.135653 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e7da16-43f7-427f-9e0b-ec0462c091b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:25.137363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e7da16-43f7-427f-9e0b-ec0462c091b6 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:25.149529 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1179/2350] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:25.153795 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f27df6-7935-4361-a172-e50bb26718a4 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:25.164397 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f27df6-7935-4361-a172-e50bb26718a4 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:25.169072 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdeb9ac-19e4-4752-9b38-e75363d05992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:25.169728 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdeb9ac-19e4-4752-9b38-e75363d05992 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:25.170001 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdeb9ac-19e4-4752-9b38-e75363d05992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:25.170282 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdeb9ac-19e4-4752-9b38-e75363d05992 admin admin] PATH_INFO: `/v3/projects/9560c874d7b14664856d77ab364670ad/users/7c2b8643f3784d10848d26afaa174ceb/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:25.185778 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdeb9ac-19e4-4752-9b38-e75363d05992 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9560c874d7b14664856d77ab364670ad, user_id=7c2b8643f3784d10848d26afaa174ceb, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:25.188724 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdeb9ac-19e4-4752-9b38-e75363d05992 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:25.203144 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1172/2351] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:33:25 2022] PUT /identity/v3/projects/9560c874d7b14664856d77ab364670ad/users/7c2b8643f3784d10848d26afaa174ceb/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:33:25.213777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e0722-359d-444e-a8c0-6ac61c5c3502 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:25.214070 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e0722-359d-444e-a8c0-6ac61c5c3502 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:25.214352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e0722-359d-444e-a8c0-6ac61c5c3502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:25.242759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-041e0722-359d-444e-a8c0-6ac61c5c3502 None None] MFA Rules not processed for user `7c2b8643f3784d10848d26afaa174ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:25.244006 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-041e0722-359d-444e-a8c0-6ac61c5c3502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:25.256867 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:25.315367 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1180/2352] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:25.320214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdeb9ac-19e4-4752-9b38-e75363d05992 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:25.387766 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdeb9ac-19e4-4752-9b38-e75363d05992 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:25.392530 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a7009b-7801-4ced-9079-e4bdad963eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:25.393150 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7009b-7801-4ced-9079-e4bdad963eb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:25.393423 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7009b-7801-4ced-9079-e4bdad963eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:25.393691 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7009b-7801-4ced-9079-e4bdad963eb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:25.394321 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a7009b-7801-4ced-9079-e4bdad963eb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:25.396630 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a7009b-7801-4ced-9079-e4bdad963eb5 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:25.413479 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:25.435541 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1173/2353] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:33:25 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:33:25.436792 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e0722-359d-444e-a8c0-6ac61c5c3502 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:25.449203 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e0722-359d-444e-a8c0-6ac61c5c3502 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:25.512608 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e734a9-1059-47d4-a0a9-a5bd4ce472e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:25.513245 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e734a9-1059-47d4-a0a9-a5bd4ce472e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:25.513502 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e734a9-1059-47d4-a0a9-a5bd4ce472e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:25.513771 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e734a9-1059-47d4-a0a9-a5bd4ce472e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:25.514394 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e734a9-1059-47d4-a0a9-a5bd4ce472e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:25.516395 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e734a9-1059-47d4-a0a9-a5bd4ce472e7 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:25.540156 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1181/2354] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:33:25 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:25.544376 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a7009b-7801-4ced-9079-e4bdad963eb5 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:25.555304 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a7009b-7801-4ced-9079-e4bdad963eb5 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:25.560375 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dae7da-9ef1-4540-a7cf-7fe3a3c69427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:25.560957 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dae7da-9ef1-4540-a7cf-7fe3a3c69427 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:25.561235 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dae7da-9ef1-4540-a7cf-7fe3a3c69427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:25.561506 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dae7da-9ef1-4540-a7cf-7fe3a3c69427 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:25.562143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dae7da-9ef1-4540-a7cf-7fe3a3c69427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:25.563751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dae7da-9ef1-4540-a7cf-7fe3a3c69427 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:25.574971 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1174/2355] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:25.579487 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e734a9-1059-47d4-a0a9-a5bd4ce472e7 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:25.590633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e734a9-1059-47d4-a0a9-a5bd4ce472e7 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:25.595321 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780cf8d9-b149-46a0-8a7e-7ac567637218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:25.595943 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780cf8d9-b149-46a0-8a7e-7ac567637218 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:25.596239 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780cf8d9-b149-46a0-8a7e-7ac567637218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:25.596524 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780cf8d9-b149-46a0-8a7e-7ac567637218 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:25.597187 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780cf8d9-b149-46a0-8a7e-7ac567637218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:25.598854 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780cf8d9-b149-46a0-8a7e-7ac567637218 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:25.612640 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1182/2356] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:25.616416 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dae7da-9ef1-4540-a7cf-7fe3a3c69427 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:25.627108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dae7da-9ef1-4540-a7cf-7fe3a3c69427 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:25.632978 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9d81d-9a85-44c7-86a6-eb7a3cffeaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:25.633828 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9d81d-9a85-44c7-86a6-eb7a3cffeaa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:25.634103 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9d81d-9a85-44c7-86a6-eb7a3cffeaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:25.634373 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9d81d-9a85-44c7-86a6-eb7a3cffeaa5 admin admin] PATH_INFO: `/v3/projects/2df07f1454c64ade8250a4cb6119e5fa/users/21b11b549f39429393d6594b47299631/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:25.650916 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9d81d-9a85-44c7-86a6-eb7a3cffeaa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2df07f1454c64ade8250a4cb6119e5fa, user_id=21b11b549f39429393d6594b47299631, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:25.653649 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9d81d-9a85-44c7-86a6-eb7a3cffeaa5 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:25.667784 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1175/2357] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:33:25 2022] PUT /identity/v3/projects/2df07f1454c64ade8250a4cb6119e5fa/users/21b11b549f39429393d6594b47299631/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:25.678460 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef026d-f351-4fe4-bda8-acef1ccb0c39 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:25.678747 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef026d-f351-4fe4-bda8-acef1ccb0c39 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:25.679049 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef026d-f351-4fe4-bda8-acef1ccb0c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:25.708858 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-c0ef026d-f351-4fe4-bda8-acef1ccb0c39 None None] MFA Rules not processed for user `21b11b549f39429393d6594b47299631`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:25.710067 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-c0ef026d-f351-4fe4-bda8-acef1ccb0c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:25.777853 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1183/2358] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:25.782769 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9d81d-9a85-44c7-86a6-eb7a3cffeaa5 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:25.858059 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9d81d-9a85-44c7-86a6-eb7a3cffeaa5 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:25.866097 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884822d-7869-4122-9227-e359519f4dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:25.866929 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884822d-7869-4122-9227-e359519f4dbf admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:25.867206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884822d-7869-4122-9227-e359519f4dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:25.867540 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884822d-7869-4122-9227-e359519f4dbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:25.868579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f884822d-7869-4122-9227-e359519f4dbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:25.871803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f884822d-7869-4122-9227-e359519f4dbf admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:25.908528 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1176/2359] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:33:25 2022] POST /identity/v3/projects => generated 387 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:25.912982 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef026d-f351-4fe4-bda8-acef1ccb0c39 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:25.924978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef026d-f351-4fe4-bda8-acef1ccb0c39 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:25.947235 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:25.991289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a98961-c70d-4ee5-a810-463e4c0b70d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:25.991944 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a98961-c70d-4ee5-a810-463e4c0b70d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:25.992241 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a98961-c70d-4ee5-a810-463e4c0b70d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:25.992552 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a98961-c70d-4ee5-a810-463e4c0b70d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:25.993155 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a98961-c70d-4ee5-a810-463e4c0b70d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:25.995107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a98961-c70d-4ee5-a810-463e4c0b70d6 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:26.020129 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1184/2360] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:33:25 2022] POST /identity/v3/users => generated 300 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:26.021630 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884822d-7869-4122-9227-e359519f4dbf admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:26.032668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884822d-7869-4122-9227-e359519f4dbf admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:26.038451 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e484f93-eff8-408d-b849-2cda7ab92a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:26.039078 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e484f93-eff8-408d-b849-2cda7ab92a0f admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:26.039360 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e484f93-eff8-408d-b849-2cda7ab92a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:26.039633 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e484f93-eff8-408d-b849-2cda7ab92a0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:26.040461 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e484f93-eff8-408d-b849-2cda7ab92a0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:26.042072 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e484f93-eff8-408d-b849-2cda7ab92a0f admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:26.053758 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1177/2361] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:26.058095 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a98961-c70d-4ee5-a810-463e4c0b70d6 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:26.072949 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a98961-c70d-4ee5-a810-463e4c0b70d6 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:26.080232 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb5370-f673-460e-98e5-b88235282e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:26.080925 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb5370-f673-460e-98e5-b88235282e23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:26.081184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb5370-f673-460e-98e5-b88235282e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:26.081449 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb5370-f673-460e-98e5-b88235282e23 admin admin] PATH_INFO: `/v3/projects/dde612319ba54273b63af4f92eb504c4/users/ea251cac2f414e63bca47f5d37f0a004/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:26.099095 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eb5370-f673-460e-98e5-b88235282e23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dde612319ba54273b63af4f92eb504c4, user_id=ea251cac2f414e63bca47f5d37f0a004, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:26.102219 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eb5370-f673-460e-98e5-b88235282e23 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:26.115777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1185/2362] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:33:26 2022] PUT /identity/v3/projects/dde612319ba54273b63af4f92eb504c4/users/ea251cac2f414e63bca47f5d37f0a004/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:26.120115 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e484f93-eff8-408d-b849-2cda7ab92a0f admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:26.130835 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e484f93-eff8-408d-b849-2cda7ab92a0f admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:26.193093 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3435c26d-a0e1-4702-91be-5fdb2112897c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:26.193744 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435c26d-a0e1-4702-91be-5fdb2112897c admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:26.194020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435c26d-a0e1-4702-91be-5fdb2112897c admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:26.194323 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435c26d-a0e1-4702-91be-5fdb2112897c admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:26.194981 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3435c26d-a0e1-4702-91be-5fdb2112897c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:26.196648 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3435c26d-a0e1-4702-91be-5fdb2112897c admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:26.207220 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1178/2363] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:26.211688 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb5370-f673-460e-98e5-b88235282e23 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:26.222738 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb5370-f673-460e-98e5-b88235282e23 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:26.228532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a50978-356f-4713-9372-7aca488070fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:26.229202 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a50978-356f-4713-9372-7aca488070fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:26.229506 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a50978-356f-4713-9372-7aca488070fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:26.229776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a50978-356f-4713-9372-7aca488070fe admin admin] PATH_INFO: `/v3/projects/dde612319ba54273b63af4f92eb504c4/users/ea251cac2f414e63bca47f5d37f0a004/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:26.231783 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a50978-356f-4713-9372-7aca488070fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=dde612319ba54273b63af4f92eb504c4, user_id=ea251cac2f414e63bca47f5d37f0a004, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:26.234516 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a50978-356f-4713-9372-7aca488070fe admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:26.245286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1186/2364] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:33:26 2022] PUT /identity/v3/projects/dde612319ba54273b63af4f92eb504c4/users/ea251cac2f414e63bca47f5d37f0a004/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:26.256637 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ef597-d351-4c4f-874d-848aa899027e None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:26.256920 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ef597-d351-4c4f-874d-848aa899027e None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:26.257192 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ef597-d351-4c4f-874d-848aa899027e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:26.285071 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-cd3ef597-d351-4c4f-874d-848aa899027e None None] MFA Rules not processed for user `ea251cac2f414e63bca47f5d37f0a004`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:26.286563 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-cd3ef597-d351-4c4f-874d-848aa899027e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:26.303548 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:26.365781 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1179/2365] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:26 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:26.376660 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a50978-356f-4713-9372-7aca488070fe admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:26.378581 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-50a50978-356f-4713-9372-7aca488070fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:26.437594 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a50978-356f-4713-9372-7aca488070fe admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:26.441576 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ecdf66-db43-467f-b185-a850c955112a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:26.442356 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecdf66-db43-467f-b185-a850c955112a service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:26.442636 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecdf66-db43-467f-b185-a850c955112a service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:26.442905 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecdf66-db43-467f-b185-a850c955112a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:26.452322 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ecdf66-db43-467f-b185-a850c955112a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:26.453787 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ecdf66-db43-467f-b185-a850c955112a service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:26.463805 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1187/2366] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:26 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:33:27.054385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ef597-d351-4c4f-874d-848aa899027e None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:27.056195 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-cd3ef597-d351-4c4f-874d-848aa899027e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:27.071942 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ef597-d351-4c4f-874d-848aa899027e None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:27.075829 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1377f57-fe24-4ca8-b9f5-ed63df10acdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:27.076498 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1377f57-fe24-4ca8-b9f5-ed63df10acdd service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:27.076780 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1377f57-fe24-4ca8-b9f5-ed63df10acdd service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:27.077053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1377f57-fe24-4ca8-b9f5-ed63df10acdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:27.138437 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1377f57-fe24-4ca8-b9f5-ed63df10acdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:27.139948 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1377f57-fe24-4ca8-b9f5-ed63df10acdd service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:27.149449 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1180/2367] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:27 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:33:28.415834 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d459904-39d3-4e60-abe4-a0e87a2b9d77 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:28.416261 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d459904-39d3-4e60-abe4-a0e87a2b9d77 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:28.416454 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d459904-39d3-4e60-abe4-a0e87a2b9d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:28.436445 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-7d459904-39d3-4e60-abe4-a0e87a2b9d77 None None] MFA Rules not processed for user `d3538271ebfd41aabe9f3da85feb7bca`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:28.437672 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-7d459904-39d3-4e60-abe4-a0e87a2b9d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:28.497231 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1188/2368] 10.222.0.18 () {58 vars in 967 bytes} [Tue Aug 23 04:33:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:28.513660 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1377f57-fe24-4ca8-b9f5-ed63df10acdd service cinder] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:28.515073 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-a1377f57-fe24-4ca8-b9f5-ed63df10acdd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:28.586210 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1377f57-fe24-4ca8-b9f5-ed63df10acdd service cinder] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:28.591276 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b53ec4-9940-4a7b-b256-000c4fb44c0f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f1418fd211a448da1a50175dce13b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:28.591889 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b53ec4-9940-4a7b-b256-000c4fb44c0f service swift] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:28.592194 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b53ec4-9940-4a7b-b256-000c4fb44c0f service swift] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:28.592465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b53ec4-9940-4a7b-b256-000c4fb44c0f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:28.601086 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b53ec4-9940-4a7b-b256-000c4fb44c0f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:28.602669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b53ec4-9940-4a7b-b256-000c4fb44c0f service swift] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:28.610632 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1181/2369] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Aug 23 04:33:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 23 04:33:28.943747 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d459904-39d3-4e60-abe4-a0e87a2b9d77 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:28.945568 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-7d459904-39d3-4e60-abe4-a0e87a2b9d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:28.972434 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:29.007303 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d459904-39d3-4e60-abe4-a0e87a2b9d77 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:29.011207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ed631-531b-4361-afa9-883925cedf64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:29.011829 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ed631-531b-4361-afa9-883925cedf64 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:29.012134 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ed631-531b-4361-afa9-883925cedf64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:29.012403 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ed631-531b-4361-afa9-883925cedf64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:29.014347 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-1c0ed631-531b-4361-afa9-883925cedf64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:29.077775 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ed631-531b-4361-afa9-883925cedf64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:29.079215 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ed631-531b-4361-afa9-883925cedf64 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:29.089109 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1189/2370] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:33:31.261960 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b53ec4-9940-4a7b-b256-000c4fb44c0f service swift] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:31.271782 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b53ec4-9940-4a7b-b256-000c4fb44c0f service swift] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:31.276786 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82eef3-a9c4-4fc2-b994-b7c4ae65e1ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:31.277559 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82eef3-a9c4-4fc2-b994-b7c4ae65e1ce service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:31.277957 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82eef3-a9c4-4fc2-b994-b7c4ae65e1ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:31.278342 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82eef3-a9c4-4fc2-b994-b7c4ae65e1ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:31.328519 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:31.347838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab82eef3-a9c4-4fc2-b994-b7c4ae65e1ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:31.349353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab82eef3-a9c4-4fc2-b994-b7c4ae65e1ce service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:31.360150 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1182/2371] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:33:32.932360 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ed631-531b-4361-afa9-883925cedf64 service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:32.934069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-1c0ed631-531b-4361-afa9-883925cedf64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:32.948498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ed631-531b-4361-afa9-883925cedf64 service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:32.953875 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003c6579-8439-4bf2-b486-d19fb76cb6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:32.954524 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c6579-8439-4bf2-b486-d19fb76cb6dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:32.954802 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c6579-8439-4bf2-b486-d19fb76cb6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:32.955076 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c6579-8439-4bf2-b486-d19fb76cb6dc admin admin] PATH_INFO: `/v3/users/658ce95360454ea1a96a22af310e5a4f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:32.956502 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003c6579-8439-4bf2-b486-d19fb76cb6dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=658ce95360454ea1a96a22af310e5a4f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:32.958435 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003c6579-8439-4bf2-b486-d19fb76cb6dc admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:33.034038 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-003c6579-8439-4bf2-b486-d19fb76cb6dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '658ce95360454ea1a96a22af310e5a4f'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:33.035143 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-003c6579-8439-4bf2-b486-d19fb76cb6dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '658ce95360454ea1a96a22af310e5a4f'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:33.045491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-003c6579-8439-4bf2-b486-d19fb76cb6dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '658ce95360454ea1a96a22af310e5a4f'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:33.065349 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-003c6579-8439-4bf2-b486-d19fb76cb6dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '658ce95360454ea1a96a22af310e5a4f'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:33.079201 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-003c6579-8439-4bf2-b486-d19fb76cb6dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '658ce95360454ea1a96a22af310e5a4f'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:33.087126 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1190/2372] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:33:32 2022] DELETE /identity/v3/users/658ce95360454ea1a96a22af310e5a4f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:33.279397 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82eef3-a9c4-4fc2-b994-b7c4ae65e1ce service cinder] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:33.294317 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82eef3-a9c4-4fc2-b994-b7c4ae65e1ce service cinder] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:33.368712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968691e5-ff93-4fae-afb7-32e84bea1ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:33.369358 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968691e5-ff93-4fae-afb7-32e84bea1ef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:33.369639 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968691e5-ff93-4fae-afb7-32e84bea1ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:33.369915 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968691e5-ff93-4fae-afb7-32e84bea1ef4 admin admin] PATH_INFO: `/v3/projects/620abd3a734b4502afa795d2cc58dadf` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:33.371033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968691e5-ff93-4fae-afb7-32e84bea1ef4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=620abd3a734b4502afa795d2cc58dadf)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:33.373002 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968691e5-ff93-4fae-afb7-32e84bea1ef4 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:33.422050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-968691e5-ff93-4fae-afb7-32e84bea1ef4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '620abd3a734b4502afa795d2cc58dadf', 'request_id': 'req-968691e5-ff93-4fae-afb7-32e84bea1ef4'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:33.428256 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-968691e5-ff93-4fae-afb7-32e84bea1ef4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '620abd3a734b4502afa795d2cc58dadf', 'request_id': 'req-968691e5-ff93-4fae-afb7-32e84bea1ef4'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:33.436403 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-968691e5-ff93-4fae-afb7-32e84bea1ef4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '620abd3a734b4502afa795d2cc58dadf', 'request_id': 'req-968691e5-ff93-4fae-afb7-32e84bea1ef4'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:33.448228 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-968691e5-ff93-4fae-afb7-32e84bea1ef4 admin admin] The token cache is being invalidate because project 620abd3a734b4502afa795d2cc58dadf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:33:33.448543 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-968691e5-ff93-4fae-afb7-32e84bea1ef4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:33.449212 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-968691e5-ff93-4fae-afb7-32e84bea1ef4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:33.450754 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1183/2373] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:33:33 2022] DELETE /identity/v3/projects/620abd3a734b4502afa795d2cc58dadf => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:33.456946 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7f1a7-5f09-4b9b-83d3-7afbd32ae154 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:33.457234 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7f1a7-5f09-4b9b-83d3-7afbd32ae154 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:33.457508 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7f1a7-5f09-4b9b-83d3-7afbd32ae154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:33.480062 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-88b7f1a7-5f09-4b9b-83d3-7afbd32ae154 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:33.481488 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-88b7f1a7-5f09-4b9b-83d3-7afbd32ae154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:33.522997 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:33.543766 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1191/2374] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:33.551360 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641fbfa8-19c5-44e5-b567-7761e82ad32f None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:33.551648 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641fbfa8-19c5-44e5-b567-7761e82ad32f None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:33.551945 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641fbfa8-19c5-44e5-b567-7761e82ad32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:33.576635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-641fbfa8-19c5-44e5-b567-7761e82ad32f None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:33.577860 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-641fbfa8-19c5-44e5-b567-7761e82ad32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:33.586849 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b7f1a7-5f09-4b9b-83d3-7afbd32ae154 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:33.588230 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1184/2375] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:33.588397 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-88b7f1a7-5f09-4b9b-83d3-7afbd32ae154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:33.591981 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641fbfa8-19c5-44e5-b567-7761e82ad32f None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:33.602446 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641fbfa8-19c5-44e5-b567-7761e82ad32f None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:33.604268 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b7f1a7-5f09-4b9b-83d3-7afbd32ae154 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:33.608004 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01924ac5-7629-41e7-aaa4-5205bf589814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:33.608618 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01924ac5-7629-41e7-aaa4-5205bf589814 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:33.608899 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01924ac5-7629-41e7-aaa4-5205bf589814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:33.609040 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee28b7c5-b8ff-4403-8393-0a1eee502a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:33.609199 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01924ac5-7629-41e7-aaa4-5205bf589814 admin admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:33.609673 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28b7c5-b8ff-4403-8393-0a1eee502a63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:33.609817 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01924ac5-7629-41e7-aaa4-5205bf589814 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:33:33.609981 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28b7c5-b8ff-4403-8393-0a1eee502a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:33.610081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01924ac5-7629-41e7-aaa4-5205bf589814 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:33.610217 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28b7c5-b8ff-4403-8393-0a1eee502a63 admin admin] PATH_INFO: `/v3/users/7c2b8643f3784d10848d26afaa174ceb` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:33.611331 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee28b7c5-b8ff-4403-8393-0a1eee502a63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c2b8643f3784d10848d26afaa174ceb)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:33.611679 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01924ac5-7629-41e7-aaa4-5205bf589814 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:33.613323 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee28b7c5-b8ff-4403-8393-0a1eee502a63 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:33.627936 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1185/2376] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:33:33.629008 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01924ac5-7629-41e7-aaa4-5205bf589814 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:33.639160 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01924ac5-7629-41e7-aaa4-5205bf589814 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:33.644393 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32730f-5bdb-4aa7-8ed7-d35e58a7b3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:33.645121 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32730f-5bdb-4aa7-8ed7-d35e58a7b3ae admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:33.645398 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32730f-5bdb-4aa7-8ed7-d35e58a7b3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:33.645670 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32730f-5bdb-4aa7-8ed7-d35e58a7b3ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:33.646406 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b32730f-5bdb-4aa7-8ed7-d35e58a7b3ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:33.648832 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b32730f-5bdb-4aa7-8ed7-d35e58a7b3ae admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:33.664410 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ee28b7c5-b8ff-4403-8393-0a1eee502a63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c2b8643f3784d10848d26afaa174ceb'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:33.665111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ee28b7c5-b8ff-4403-8393-0a1eee502a63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c2b8643f3784d10848d26afaa174ceb'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:33.673021 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ee28b7c5-b8ff-4403-8393-0a1eee502a63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c2b8643f3784d10848d26afaa174ceb'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:33.678963 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1186/2377] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:33:33 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:33:33.683476 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32730f-5bdb-4aa7-8ed7-d35e58a7b3ae admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:33.685237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-4b32730f-5bdb-4aa7-8ed7-d35e58a7b3ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:33.688079 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ee28b7c5-b8ff-4403-8393-0a1eee502a63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c2b8643f3784d10848d26afaa174ceb'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:33.697281 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ee28b7c5-b8ff-4403-8393-0a1eee502a63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c2b8643f3784d10848d26afaa174ceb'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:33.704513 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1192/2378] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:33:33 2022] DELETE /identity/v3/users/7c2b8643f3784d10848d26afaa174ceb => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:33.709403 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd6906-52ba-4128-a9ee-0787c56c6f83 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:33.709648 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd6906-52ba-4128-a9ee-0787c56c6f83 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:33.710711 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd6906-52ba-4128-a9ee-0787c56c6f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:33.726364 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:33.731826 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-d2bd6906-52ba-4128-a9ee-0787c56c6f83 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:33.733204 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-d2bd6906-52ba-4128-a9ee-0787c56c6f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:33.761050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32730f-5bdb-4aa7-8ed7-d35e58a7b3ae admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:33.795936 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1193/2379] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:33.807269 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd6906-52ba-4128-a9ee-0787c56c6f83 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:33.808673 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-d2bd6906-52ba-4128-a9ee-0787c56c6f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:33.821229 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29b85b-1645-460e-9c86-79f2664b6351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:33.821851 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29b85b-1645-460e-9c86-79f2664b6351 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:33.822131 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29b85b-1645-460e-9c86-79f2664b6351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:33.822414 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29b85b-1645-460e-9c86-79f2664b6351 admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:33.823049 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f29b85b-1645-460e-9c86-79f2664b6351 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:33.825041 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f29b85b-1645-460e-9c86-79f2664b6351 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:33.852208 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1187/2380] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:33:33 2022] POST /identity/v3/users => generated 294 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:33.856914 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29b85b-1645-460e-9c86-79f2664b6351 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:33.867109 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29b85b-1645-460e-9c86-79f2664b6351 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:33.871795 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05065fe9-e59c-42f7-ae78-8e041ae4b840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:33.871999 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd6906-52ba-4128-a9ee-0787c56c6f83 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:33.872438 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05065fe9-e59c-42f7-ae78-8e041ae4b840 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:33.872740 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05065fe9-e59c-42f7-ae78-8e041ae4b840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:33.873033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05065fe9-e59c-42f7-ae78-8e041ae4b840 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:33.873657 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05065fe9-e59c-42f7-ae78-8e041ae4b840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:33.875285 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05065fe9-e59c-42f7-ae78-8e041ae4b840 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:33.875859 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398d24f-dd68-4fe4-98e5-16d36c1f9458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:33.876500 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398d24f-dd68-4fe4-98e5-16d36c1f9458 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:33.876806 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398d24f-dd68-4fe4-98e5-16d36c1f9458 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:33.877056 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398d24f-dd68-4fe4-98e5-16d36c1f9458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:33.886461 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1188/2381] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:33.887566 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398d24f-dd68-4fe4-98e5-16d36c1f9458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:33.889566 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398d24f-dd68-4fe4-98e5-16d36c1f9458 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:33.890494 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05065fe9-e59c-42f7-ae78-8e041ae4b840 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:33.895061 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:33.900280 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05065fe9-e59c-42f7-ae78-8e041ae4b840 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:33.905768 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005fadfe-1fed-488c-898d-044b5fc55ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:33.906372 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fadfe-1fed-488c-898d-044b5fc55ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:33.906650 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fadfe-1fed-488c-898d-044b5fc55ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:33.906923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fadfe-1fed-488c-898d-044b5fc55ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:33.907562 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005fadfe-1fed-488c-898d-044b5fc55ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:33.908094 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1194/2382] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:33:33.909262 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005fadfe-1fed-488c-898d-044b5fc55ce0 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:33.919999 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1189/2383] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:33:33.924363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398d24f-dd68-4fe4-98e5-16d36c1f9458 service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:33.934031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398d24f-dd68-4fe4-98e5-16d36c1f9458 service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:33.939440 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d7ce8-7221-4163-8454-e5516ca8c88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:33.940166 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d7ce8-7221-4163-8454-e5516ca8c88e admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:33.940448 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d7ce8-7221-4163-8454-e5516ca8c88e admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:33.940717 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d7ce8-7221-4163-8454-e5516ca8c88e admin admin] PATH_INFO: `/v3/projects/f3bc2f3d1e634d918bc6a8f489fd3444/users/f505a8dca83e4138be8a043b92ebb5ec/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:33.957851 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53d7ce8-7221-4163-8454-e5516ca8c88e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3bc2f3d1e634d918bc6a8f489fd3444, user_id=f505a8dca83e4138be8a043b92ebb5ec, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:33.960807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53d7ce8-7221-4163-8454-e5516ca8c88e admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:33.976731 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1195/2384] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:33:33 2022] PUT /identity/v3/projects/f3bc2f3d1e634d918bc6a8f489fd3444/users/f505a8dca83e4138be8a043b92ebb5ec/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:33.992801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c0041-da6d-460a-8c5b-dc16819ff447 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:33.993308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c0041-da6d-460a-8c5b-dc16819ff447 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:33.993768 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c0041-da6d-460a-8c5b-dc16819ff447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:34.024760 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-986c0041-da6d-460a-8c5b-dc16819ff447 None None] MFA Rules not processed for user `f505a8dca83e4138be8a043b92ebb5ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:34.026039 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-986c0041-da6d-460a-8c5b-dc16819ff447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:34.099395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1190/2385] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:33 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:34.104469 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d7ce8-7221-4163-8454-e5516ca8c88e admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:34.167574 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d7ce8-7221-4163-8454-e5516ca8c88e admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:34.172425 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e5727-dd3d-4f73-9fbe-04baf3ae2a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:34.173029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e5727-dd3d-4f73-9fbe-04baf3ae2a73 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:34.173434 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e5727-dd3d-4f73-9fbe-04baf3ae2a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:34.174001 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e5727-dd3d-4f73-9fbe-04baf3ae2a73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:34.174696 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468e5727-dd3d-4f73-9fbe-04baf3ae2a73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:34.176651 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468e5727-dd3d-4f73-9fbe-04baf3ae2a73 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:34.189878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c0041-da6d-460a-8c5b-dc16819ff447 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:34.191343 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-986c0041-da6d-460a-8c5b-dc16819ff447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:34.211957 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1196/2386] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:33:34 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:34.215382 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e5727-dd3d-4f73-9fbe-04baf3ae2a73 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:34.217197 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c0041-da6d-460a-8c5b-dc16819ff447 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:34.254640 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:34.286465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff06d0-a339-4f68-80fb-a23a9d87e5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:34.287167 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff06d0-a339-4f68-80fb-a23a9d87e5e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:34.287412 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff06d0-a339-4f68-80fb-a23a9d87e5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:34.287645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff06d0-a339-4f68-80fb-a23a9d87e5e1 admin admin] PATH_INFO: `/v3/projects/9560c874d7b14664856d77ab364670ad` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:34.288115 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e5727-dd3d-4f73-9fbe-04baf3ae2a73 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:34.289428 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ff06d0-a339-4f68-80fb-a23a9d87e5e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9560c874d7b14664856d77ab364670ad)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:34.291370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ff06d0-a339-4f68-80fb-a23a9d87e5e1 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:34.295339 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9a8fe-1e81-4606-a9bc-3fbaeb3db29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:34.296386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9a8fe-1e81-4606-a9bc-3fbaeb3db29c admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:34.296665 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9a8fe-1e81-4606-a9bc-3fbaeb3db29c admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:34.296936 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9a8fe-1e81-4606-a9bc-3fbaeb3db29c admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:34.297562 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b9a8fe-1e81-4606-a9bc-3fbaeb3db29c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:34.299532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b9a8fe-1e81-4606-a9bc-3fbaeb3db29c admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:34.326221 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1197/2387] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:33:34 2022] POST /identity/v3/users => generated 299 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:33:34.330424 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9a8fe-1e81-4606-a9bc-3fbaeb3db29c admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:34.340505 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9a8fe-1e81-4606-a9bc-3fbaeb3db29c admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:34.346580 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f7ff06d0-a339-4f68-80fb-a23a9d87e5e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9560c874d7b14664856d77ab364670ad', 'request_id': 'req-f7ff06d0-a339-4f68-80fb-a23a9d87e5e1'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:34.355689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f7ff06d0-a339-4f68-80fb-a23a9d87e5e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9560c874d7b14664856d77ab364670ad', 'request_id': 'req-f7ff06d0-a339-4f68-80fb-a23a9d87e5e1'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:34.367014 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f7ff06d0-a339-4f68-80fb-a23a9d87e5e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9560c874d7b14664856d77ab364670ad', 'request_id': 'req-f7ff06d0-a339-4f68-80fb-a23a9d87e5e1'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:34.378791 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f7ff06d0-a339-4f68-80fb-a23a9d87e5e1 admin admin] The token cache is being invalidate because project 9560c874d7b14664856d77ab364670ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:33:34.379077 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f7ff06d0-a339-4f68-80fb-a23a9d87e5e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:34.379738 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f7ff06d0-a339-4f68-80fb-a23a9d87e5e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:34.382047 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1191/2388] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:33:34 2022] DELETE /identity/v3/projects/9560c874d7b14664856d77ab364670ad => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:33:34.385580 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff06d0-a339-4f68-80fb-a23a9d87e5e1 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:34.387093 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-f7ff06d0-a339-4f68-80fb-a23a9d87e5e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:34.397515 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:34.408921 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e424a-a8cd-41db-8396-bd420936f533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:34.409562 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e424a-a8cd-41db-8396-bd420936f533 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:34.409843 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e424a-a8cd-41db-8396-bd420936f533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:34.410115 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e424a-a8cd-41db-8396-bd420936f533 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:34.410765 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e424a-a8cd-41db-8396-bd420936f533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:34.412464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e424a-a8cd-41db-8396-bd420936f533 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:34.423193 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1198/2389] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:34.427262 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e424a-a8cd-41db-8396-bd420936f533 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:34.428950 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a07e424a-a8cd-41db-8396-bd420936f533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:34.441261 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e424a-a8cd-41db-8396-bd420936f533 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:34.446077 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827dee1-b66f-4416-b3a6-1db0a5e9a078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:34.446726 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827dee1-b66f-4416-b3a6-1db0a5e9a078 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:34.447069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827dee1-b66f-4416-b3a6-1db0a5e9a078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:34.447281 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827dee1-b66f-4416-b3a6-1db0a5e9a078 admin admin] PATH_INFO: `/v3/projects/30772bbd4cb74cef96dc2e5b4c100491/users/328a3e81f67a4917b77fea4091cfd6a7/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:34.455989 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff06d0-a339-4f68-80fb-a23a9d87e5e1 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:34.461421 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb6158-59b8-4e52-8183-fe2fa19daff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:34.462064 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb6158-59b8-4e52-8183-fe2fa19daff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:34.462344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb6158-59b8-4e52-8183-fe2fa19daff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:34.462644 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb6158-59b8-4e52-8183-fe2fa19daff8 admin admin] PATH_INFO: `/v3/users/21b11b549f39429393d6594b47299631` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:34.463678 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827dee1-b66f-4416-b3a6-1db0a5e9a078 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30772bbd4cb74cef96dc2e5b4c100491, user_id=328a3e81f67a4917b77fea4091cfd6a7, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:34.463795 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb6158-59b8-4e52-8183-fe2fa19daff8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21b11b549f39429393d6594b47299631)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:34.465908 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb6158-59b8-4e52-8183-fe2fa19daff8 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:34.467550 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827dee1-b66f-4416-b3a6-1db0a5e9a078 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:34.482107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1199/2390] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:33:34 2022] PUT /identity/v3/projects/30772bbd4cb74cef96dc2e5b4c100491/users/328a3e81f67a4917b77fea4091cfd6a7/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:33:34.486115 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827dee1-b66f-4416-b3a6-1db0a5e9a078 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:34.519726 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-6abb6158-59b8-4e52-8183-fe2fa19daff8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21b11b549f39429393d6594b47299631'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:34.527644 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-6abb6158-59b8-4e52-8183-fe2fa19daff8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21b11b549f39429393d6594b47299631'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:34.536075 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:34.542007 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-6abb6158-59b8-4e52-8183-fe2fa19daff8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21b11b549f39429393d6594b47299631'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:34.543053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-6abb6158-59b8-4e52-8183-fe2fa19daff8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21b11b549f39429393d6594b47299631'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:34.549537 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827dee1-b66f-4416-b3a6-1db0a5e9a078 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:34.552352 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-6abb6158-59b8-4e52-8183-fe2fa19daff8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21b11b549f39429393d6594b47299631'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:34.555658 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca082c-d22d-4b96-b82d-3e8243578eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:34.556323 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca082c-d22d-4b96-b82d-3e8243578eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:34.556612 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca082c-d22d-4b96-b82d-3e8243578eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:34.556874 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca082c-d22d-4b96-b82d-3e8243578eb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:34.557514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca082c-d22d-4b96-b82d-3e8243578eb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:34.558488 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1192/2391] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:33:34 2022] DELETE /identity/v3/users/21b11b549f39429393d6594b47299631 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:34.561487 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca082c-d22d-4b96-b82d-3e8243578eb6 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:34.574536 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1200/2392] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:33:34.578743 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb6158-59b8-4e52-8183-fe2fa19daff8 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:34.580389 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-6abb6158-59b8-4e52-8183-fe2fa19daff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:34.651640 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb6158-59b8-4e52-8183-fe2fa19daff8 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:34.656312 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0db80-b7ea-4cce-88e6-3a3e68bf95cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:34.657008 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0db80-b7ea-4cce-88e6-3a3e68bf95cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:34.657310 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0db80-b7ea-4cce-88e6-3a3e68bf95cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:34.657566 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0db80-b7ea-4cce-88e6-3a3e68bf95cf admin admin] PATH_INFO: `/v3/projects/30772bbd4cb74cef96dc2e5b4c100491/users/328a3e81f67a4917b77fea4091cfd6a7/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:34.659573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a0db80-b7ea-4cce-88e6-3a3e68bf95cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=30772bbd4cb74cef96dc2e5b4c100491, user_id=328a3e81f67a4917b77fea4091cfd6a7, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:34.662330 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a0db80-b7ea-4cce-88e6-3a3e68bf95cf admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:34.675753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1193/2393] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:33:34 2022] PUT /identity/v3/projects/30772bbd4cb74cef96dc2e5b4c100491/users/328a3e81f67a4917b77fea4091cfd6a7/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:34.687039 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d4383-2ee8-4a8c-a071-b5222245c5ec None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:34.687304 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d4383-2ee8-4a8c-a071-b5222245c5ec None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:34.687676 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d4383-2ee8-4a8c-a071-b5222245c5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:34.719853 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-3e6d4383-2ee8-4a8c-a071-b5222245c5ec None None] MFA Rules not processed for user `328a3e81f67a4917b77fea4091cfd6a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:34.721183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-3e6d4383-2ee8-4a8c-a071-b5222245c5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:34.790208 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1201/2394] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:34 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:34.803493 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0db80-b7ea-4cce-88e6-3a3e68bf95cf admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:34.805799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-a3a0db80-b7ea-4cce-88e6-3a3e68bf95cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:34.844981 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d4383-2ee8-4a8c-a071-b5222245c5ec None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:34.846627 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-3e6d4383-2ee8-4a8c-a071-b5222245c5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:34.873787 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0db80-b7ea-4cce-88e6-3a3e68bf95cf admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:34.877450 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f98bae-0d66-4ebd-949c-0268f92f10e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:34.878068 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f98bae-0d66-4ebd-949c-0268f92f10e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:34.878346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f98bae-0d66-4ebd-949c-0268f92f10e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:34.878619 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f98bae-0d66-4ebd-949c-0268f92f10e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:34.888948 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f98bae-0d66-4ebd-949c-0268f92f10e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:34.890405 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f98bae-0d66-4ebd-949c-0268f92f10e1 service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:34.900639 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1194/2395] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:34 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:33:34.901990 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:34.921932 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d4383-2ee8-4a8c-a071-b5222245c5ec None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:34.926952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d12c0-46f0-4069-9e26-d9e7b0cf07bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:34.927089 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d12c0-46f0-4069-9e26-d9e7b0cf07bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:34.927318 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d12c0-46f0-4069-9e26-d9e7b0cf07bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:34.927585 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d12c0-46f0-4069-9e26-d9e7b0cf07bd admin admin] PATH_INFO: `/v3/projects/2df07f1454c64ade8250a4cb6119e5fa` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:34.928950 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d12c0-46f0-4069-9e26-d9e7b0cf07bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2df07f1454c64ade8250a4cb6119e5fa)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:34.929612 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f98bae-0d66-4ebd-949c-0268f92f10e1 service cinder] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:34.930899 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d12c0-46f0-4069-9e26-d9e7b0cf07bd admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:34.939950 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f98bae-0d66-4ebd-949c-0268f92f10e1 service cinder] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:34.944453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ad5c3-7f16-4992-aeb1-7c60d2533fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:34.945119 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ad5c3-7f16-4992-aeb1-7c60d2533fcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:34.945381 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ad5c3-7f16-4992-aeb1-7c60d2533fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:34.945647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ad5c3-7f16-4992-aeb1-7c60d2533fcd admin admin] PATH_INFO: `/v3/users/f505a8dca83e4138be8a043b92ebb5ec` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:34.946774 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ad5c3-7f16-4992-aeb1-7c60d2533fcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f505a8dca83e4138be8a043b92ebb5ec)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:34.948762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ad5c3-7f16-4992-aeb1-7c60d2533fcd admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:34.980170 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-2a9d12c0-46f0-4069-9e26-d9e7b0cf07bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2df07f1454c64ade8250a4cb6119e5fa', 'request_id': 'req-2a9d12c0-46f0-4069-9e26-d9e7b0cf07bd'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:34.987111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-2a9d12c0-46f0-4069-9e26-d9e7b0cf07bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2df07f1454c64ade8250a4cb6119e5fa', 'request_id': 'req-2a9d12c0-46f0-4069-9e26-d9e7b0cf07bd'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:34.994478 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-2a9d12c0-46f0-4069-9e26-d9e7b0cf07bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2df07f1454c64ade8250a4cb6119e5fa', 'request_id': 'req-2a9d12c0-46f0-4069-9e26-d9e7b0cf07bd'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:35.002135 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-1b3ad5c3-7f16-4992-aeb1-7c60d2533fcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f505a8dca83e4138be8a043b92ebb5ec'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:35.005630 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-2a9d12c0-46f0-4069-9e26-d9e7b0cf07bd admin admin] The token cache is being invalidate because project 2df07f1454c64ade8250a4cb6119e5fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:33:35.006062 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-2a9d12c0-46f0-4069-9e26-d9e7b0cf07bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:35.006813 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-2a9d12c0-46f0-4069-9e26-d9e7b0cf07bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:35.008820 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1202/2396] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:33:34 2022] DELETE /identity/v3/projects/2df07f1454c64ade8250a4cb6119e5fa => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:33:35.012676 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-1b3ad5c3-7f16-4992-aeb1-7c60d2533fcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f505a8dca83e4138be8a043b92ebb5ec'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:35.012848 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d12c0-46f0-4069-9e26-d9e7b0cf07bd admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:35.014285 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-2a9d12c0-46f0-4069-9e26-d9e7b0cf07bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:35.025645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-1b3ad5c3-7f16-4992-aeb1-7c60d2533fcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f505a8dca83e4138be8a043b92ebb5ec'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:35.026387 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-1b3ad5c3-7f16-4992-aeb1-7c60d2533fcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f505a8dca83e4138be8a043b92ebb5ec'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:35.029153 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:35.035233 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-1b3ad5c3-7f16-4992-aeb1-7c60d2533fcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f505a8dca83e4138be8a043b92ebb5ec'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:35.042557 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1195/2397] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:33:34 2022] DELETE /identity/v3/users/f505a8dca83e4138be8a043b92ebb5ec => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:35.046604 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1c01e-4851-482a-95dd-e47f939112cd None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:35.046894 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1c01e-4851-482a-95dd-e47f939112cd None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:35.047168 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1c01e-4851-482a-95dd-e47f939112cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:35.066109 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-9af1c01e-4851-482a-95dd-e47f939112cd None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:35.067337 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-9af1c01e-4851-482a-95dd-e47f939112cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:35.082579 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d12c0-46f0-4069-9e26-d9e7b0cf07bd admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:35.126970 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1196/2398] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:35.137095 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1c01e-4851-482a-95dd-e47f939112cd None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:35.138035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc44449a-2bb7-4a47-9f3c-85197e8707f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:35.138490 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-9af1c01e-4851-482a-95dd-e47f939112cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:35.138699 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44449a-2bb7-4a47-9f3c-85197e8707f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:35.138990 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44449a-2bb7-4a47-9f3c-85197e8707f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:35.139255 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44449a-2bb7-4a47-9f3c-85197e8707f0 admin admin] PATH_INFO: `/v3/users/ea251cac2f414e63bca47f5d37f0a004` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:35.140406 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc44449a-2bb7-4a47-9f3c-85197e8707f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea251cac2f414e63bca47f5d37f0a004)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:35.142436 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc44449a-2bb7-4a47-9f3c-85197e8707f0 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:35.165171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:35.198863 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-dc44449a-2bb7-4a47-9f3c-85197e8707f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea251cac2f414e63bca47f5d37f0a004'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:35.199513 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-dc44449a-2bb7-4a47-9f3c-85197e8707f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea251cac2f414e63bca47f5d37f0a004'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:35.207394 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1c01e-4851-482a-95dd-e47f939112cd None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:35.209631 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-dc44449a-2bb7-4a47-9f3c-85197e8707f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea251cac2f414e63bca47f5d37f0a004'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:35.215282 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52b233-521d-405c-80ff-a72cda576c40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:35.215907 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52b233-521d-405c-80ff-a72cda576c40 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:35.216213 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52b233-521d-405c-80ff-a72cda576c40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:35.216499 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52b233-521d-405c-80ff-a72cda576c40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:35.218324 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-8e52b233-521d-405c-80ff-a72cda576c40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:35.223144 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-dc44449a-2bb7-4a47-9f3c-85197e8707f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea251cac2f414e63bca47f5d37f0a004'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:35.230636 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e52b233-521d-405c-80ff-a72cda576c40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:35.231303 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-dc44449a-2bb7-4a47-9f3c-85197e8707f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea251cac2f414e63bca47f5d37f0a004'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:35.232101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e52b233-521d-405c-80ff-a72cda576c40 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:35.237355 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1203/2399] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:33:35 2022] DELETE /identity/v3/users/ea251cac2f414e63bca47f5d37f0a004 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:33:35.241668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1197/2400] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:33:35.612784 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc44449a-2bb7-4a47-9f3c-85197e8707f0 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:35.614309 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-dc44449a-2bb7-4a47-9f3c-85197e8707f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:35.637257 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52b233-521d-405c-80ff-a72cda576c40 service neutron] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:35.638825 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-8e52b233-521d-405c-80ff-a72cda576c40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:35.653231 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52b233-521d-405c-80ff-a72cda576c40 service neutron] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:35.701581 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc44449a-2bb7-4a47-9f3c-85197e8707f0 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:35.707204 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7a629-86fb-4061-b14a-6374a2fc1e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:35.707951 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7a629-86fb-4061-b14a-6374a2fc1e6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:35.708305 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7a629-86fb-4061-b14a-6374a2fc1e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:35.708968 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7a629-86fb-4061-b14a-6374a2fc1e6c admin admin] PATH_INFO: `/v3/projects/dde612319ba54273b63af4f92eb504c4` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:35.711098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c7a629-86fb-4061-b14a-6374a2fc1e6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dde612319ba54273b63af4f92eb504c4)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:35.713004 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c7a629-86fb-4061-b14a-6374a2fc1e6c admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:35.727595 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f01178-5250-4b34-bcb6-c905bef39ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:35.728592 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f01178-5250-4b34-bcb6-c905bef39ce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:35.728878 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f01178-5250-4b34-bcb6-c905bef39ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:35.729155 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f01178-5250-4b34-bcb6-c905bef39ce8 admin admin] PATH_INFO: `/v3/projects/f3bc2f3d1e634d918bc6a8f489fd3444` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:35.732605 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f01178-5250-4b34-bcb6-c905bef39ce8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3bc2f3d1e634d918bc6a8f489fd3444)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:35.735284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f01178-5250-4b34-bcb6-c905bef39ce8 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:35.769285 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-82c7a629-86fb-4061-b14a-6374a2fc1e6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dde612319ba54273b63af4f92eb504c4', 'request_id': 'req-82c7a629-86fb-4061-b14a-6374a2fc1e6c'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:35.776015 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-82c7a629-86fb-4061-b14a-6374a2fc1e6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dde612319ba54273b63af4f92eb504c4', 'request_id': 'req-82c7a629-86fb-4061-b14a-6374a2fc1e6c'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:35.779780 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:35.785095 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-82c7a629-86fb-4061-b14a-6374a2fc1e6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dde612319ba54273b63af4f92eb504c4', 'request_id': 'req-82c7a629-86fb-4061-b14a-6374a2fc1e6c'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:35.790941 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-41f01178-5250-4b34-bcb6-c905bef39ce8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3bc2f3d1e634d918bc6a8f489fd3444', 'request_id': 'req-41f01178-5250-4b34-bcb6-c905bef39ce8'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:35.797558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-41f01178-5250-4b34-bcb6-c905bef39ce8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3bc2f3d1e634d918bc6a8f489fd3444', 'request_id': 'req-41f01178-5250-4b34-bcb6-c905bef39ce8'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:35.798557 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-82c7a629-86fb-4061-b14a-6374a2fc1e6c admin admin] The token cache is being invalidate because project dde612319ba54273b63af4f92eb504c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:33:35.798849 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-82c7a629-86fb-4061-b14a-6374a2fc1e6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:35.799616 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-82c7a629-86fb-4061-b14a-6374a2fc1e6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:35.801560 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1204/2401] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:33:35 2022] DELETE /identity/v3/projects/dde612319ba54273b63af4f92eb504c4 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:35.808587 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-41f01178-5250-4b34-bcb6-c905bef39ce8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3bc2f3d1e634d918bc6a8f489fd3444', 'request_id': 'req-41f01178-5250-4b34-bcb6-c905bef39ce8'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:35.810022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179174f9-33dd-4358-8b7f-1fb5827e38bb None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:35.810303 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179174f9-33dd-4358-8b7f-1fb5827e38bb None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:35.810588 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179174f9-33dd-4358-8b7f-1fb5827e38bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:35.821266 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-41f01178-5250-4b34-bcb6-c905bef39ce8 admin admin] The token cache is being invalidate because project f3bc2f3d1e634d918bc6a8f489fd3444 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:33:35.821558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-41f01178-5250-4b34-bcb6-c905bef39ce8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:35.822200 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-41f01178-5250-4b34-bcb6-c905bef39ce8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:35.823893 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1198/2402] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:33:35 2022] DELETE /identity/v3/projects/f3bc2f3d1e634d918bc6a8f489fd3444 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:35.828118 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f01178-5250-4b34-bcb6-c905bef39ce8 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:35.829610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-41f01178-5250-4b34-bcb6-c905bef39ce8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:35.832777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-179174f9-33dd-4358-8b7f-1fb5827e38bb None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:35.834037 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-179174f9-33dd-4358-8b7f-1fb5827e38bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:35.863269 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:35.906426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f01178-5250-4b34-bcb6-c905bef39ce8 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:35.906644 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1205/2403] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:33:35.910838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd26d1f9-4d5b-4cc0-bd2d-7bbe6a64f65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:35.911473 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd26d1f9-4d5b-4cc0-bd2d-7bbe6a64f65f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:35.911754 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd26d1f9-4d5b-4cc0-bd2d-7bbe6a64f65f admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:35.912066 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd26d1f9-4d5b-4cc0-bd2d-7bbe6a64f65f admin admin] PATH_INFO: `/v3/users/328a3e81f67a4917b77fea4091cfd6a7` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:35.913525 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd26d1f9-4d5b-4cc0-bd2d-7bbe6a64f65f admin admin] RBAC: Authorizing `identity:delete_user(user_id=328a3e81f67a4917b77fea4091cfd6a7)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:35.915103 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26323fd-9a3a-4a6a-813b-2c2fd236190c None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:35.915395 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26323fd-9a3a-4a6a-813b-2c2fd236190c None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:35.915600 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd26d1f9-4d5b-4cc0-bd2d-7bbe6a64f65f admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:35.915817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26323fd-9a3a-4a6a-813b-2c2fd236190c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:35.940845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-a26323fd-9a3a-4a6a-813b-2c2fd236190c None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:35.942185 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a26323fd-9a3a-4a6a-813b-2c2fd236190c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:35.952021 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1206/2404] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:35.954765 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26323fd-9a3a-4a6a-813b-2c2fd236190c None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:35.973483 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26323fd-9a3a-4a6a-813b-2c2fd236190c None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:35.979876 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ea1e7-0487-4b5f-9610-1f7eb5e2162c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:35.980703 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ea1e7-0487-4b5f-9610-1f7eb5e2162c admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:35.981278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ea1e7-0487-4b5f-9610-1f7eb5e2162c admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:35.981278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ea1e7-0487-4b5f-9610-1f7eb5e2162c admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:35.982042 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ea1e7-0487-4b5f-9610-1f7eb5e2162c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:33:35.982465 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ea1e7-0487-4b5f-9610-1f7eb5e2162c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:35.982605 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-cd26d1f9-4d5b-4cc0-bd2d-7bbe6a64f65f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '328a3e81f67a4917b77fea4091cfd6a7'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:35.985615 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ea1e7-0487-4b5f-9610-1f7eb5e2162c admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:35.991824 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-cd26d1f9-4d5b-4cc0-bd2d-7bbe6a64f65f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '328a3e81f67a4917b77fea4091cfd6a7'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:35.997768 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1207/2405] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:33:36.001826 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ea1e7-0487-4b5f-9610-1f7eb5e2162c admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:36.007642 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-cd26d1f9-4d5b-4cc0-bd2d-7bbe6a64f65f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '328a3e81f67a4917b77fea4091cfd6a7'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:36.008399 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-cd26d1f9-4d5b-4cc0-bd2d-7bbe6a64f65f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '328a3e81f67a4917b77fea4091cfd6a7'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:36.012085 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ea1e7-0487-4b5f-9610-1f7eb5e2162c admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:36.015390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-cd26d1f9-4d5b-4cc0-bd2d-7bbe6a64f65f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '328a3e81f67a4917b77fea4091cfd6a7'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:36.017818 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae72bfd-e4bb-4678-9235-e61e165da59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:36.018481 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae72bfd-e4bb-4678-9235-e61e165da59e admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:36.018636 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae72bfd-e4bb-4678-9235-e61e165da59e admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:36.019128 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae72bfd-e4bb-4678-9235-e61e165da59e admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:36.019841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae72bfd-e4bb-4678-9235-e61e165da59e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:36.021902 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1199/2406] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:33:35 2022] DELETE /identity/v3/users/328a3e81f67a4917b77fea4091cfd6a7 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:33:36.022031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae72bfd-e4bb-4678-9235-e61e165da59e admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:36.062664 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1208/2407] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:33:36 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:33:36.066940 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd26d1f9-4d5b-4cc0-bd2d-7bbe6a64f65f admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:36.068395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-cd26d1f9-4d5b-4cc0-bd2d-7bbe6a64f65f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:36.120561 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:36.141265 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd26d1f9-4d5b-4cc0-bd2d-7bbe6a64f65f admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:36.206087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09b62f-887f-478e-8a19-fca3fcd6e8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:36.206784 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09b62f-887f-478e-8a19-fca3fcd6e8c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:36.207067 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09b62f-887f-478e-8a19-fca3fcd6e8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:36.207339 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09b62f-887f-478e-8a19-fca3fcd6e8c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:36.208006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f09b62f-887f-478e-8a19-fca3fcd6e8c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:36.210238 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f09b62f-887f-478e-8a19-fca3fcd6e8c9 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:36.234083 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1200/2408] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:33:36 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:36.243938 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae72bfd-e4bb-4678-9235-e61e165da59e admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:36.254089 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae72bfd-e4bb-4678-9235-e61e165da59e admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:36.258932 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c49740-9f14-4cca-b37b-91853204f947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:36.259530 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c49740-9f14-4cca-b37b-91853204f947 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:36.259804 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c49740-9f14-4cca-b37b-91853204f947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:36.260104 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c49740-9f14-4cca-b37b-91853204f947 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:36.260742 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c49740-9f14-4cca-b37b-91853204f947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:36.262340 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c49740-9f14-4cca-b37b-91853204f947 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:36.273939 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1209/2409] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:36.277743 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09b62f-887f-478e-8a19-fca3fcd6e8c9 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:36.289045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09b62f-887f-478e-8a19-fca3fcd6e8c9 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:36.294029 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cffd08-ca80-4a4d-9dfc-77a4b79d0bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:36.294692 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cffd08-ca80-4a4d-9dfc-77a4b79d0bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:36.295103 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cffd08-ca80-4a4d-9dfc-77a4b79d0bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:36.295536 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cffd08-ca80-4a4d-9dfc-77a4b79d0bc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:36.296340 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cffd08-ca80-4a4d-9dfc-77a4b79d0bc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:36.298137 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cffd08-ca80-4a4d-9dfc-77a4b79d0bc3 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:36.310290 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1201/2410] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:36.314758 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c49740-9f14-4cca-b37b-91853204f947 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:36.325205 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c49740-9f14-4cca-b37b-91853204f947 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:36.326783 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cffd08-ca80-4a4d-9dfc-77a4b79d0bc3 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:36.328524 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-16cffd08-ca80-4a4d-9dfc-77a4b79d0bc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:36.330855 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a25ea-32d6-40b8-abb1-c35cc8b13dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:36.331607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a25ea-32d6-40b8-abb1-c35cc8b13dd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:36.331968 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a25ea-32d6-40b8-abb1-c35cc8b13dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:36.332364 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a25ea-32d6-40b8-abb1-c35cc8b13dd7 admin admin] PATH_INFO: `/v3/projects/74a9c3f0e6ff4d67aafb5901cea5a67b/users/c83dbb9f66ef4e8b9bf7c4cdca80863a/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:36.341511 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cffd08-ca80-4a4d-9dfc-77a4b79d0bc3 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:36.346516 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f5095-17f5-418f-8722-a7c450e0dbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:36.347746 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f5095-17f5-418f-8722-a7c450e0dbcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:36.348075 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f5095-17f5-418f-8722-a7c450e0dbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:36.348339 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f5095-17f5-418f-8722-a7c450e0dbcb admin admin] PATH_INFO: `/v3/projects/30772bbd4cb74cef96dc2e5b4c100491` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:36.349463 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f5095-17f5-418f-8722-a7c450e0dbcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=30772bbd4cb74cef96dc2e5b4c100491)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:36.350541 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a25ea-32d6-40b8-abb1-c35cc8b13dd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74a9c3f0e6ff4d67aafb5901cea5a67b, user_id=c83dbb9f66ef4e8b9bf7c4cdca80863a, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:36.352151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f5095-17f5-418f-8722-a7c450e0dbcb admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:36.353297 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a25ea-32d6-40b8-abb1-c35cc8b13dd7 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:36.369955 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1210/2411] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:33:36 2022] PUT /identity/v3/projects/74a9c3f0e6ff4d67aafb5901cea5a67b/users/c83dbb9f66ef4e8b9bf7c4cdca80863a/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:36.406197 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a63f5095-17f5-418f-8722-a7c450e0dbcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30772bbd4cb74cef96dc2e5b4c100491', 'request_id': 'req-a63f5095-17f5-418f-8722-a7c450e0dbcb'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:36.415198 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a63f5095-17f5-418f-8722-a7c450e0dbcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30772bbd4cb74cef96dc2e5b4c100491', 'request_id': 'req-a63f5095-17f5-418f-8722-a7c450e0dbcb'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:36.424245 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a63f5095-17f5-418f-8722-a7c450e0dbcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30772bbd4cb74cef96dc2e5b4c100491', 'request_id': 'req-a63f5095-17f5-418f-8722-a7c450e0dbcb'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:36.437465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a63f5095-17f5-418f-8722-a7c450e0dbcb admin admin] The token cache is being invalidate because project 30772bbd4cb74cef96dc2e5b4c100491 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:33:36.437839 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a63f5095-17f5-418f-8722-a7c450e0dbcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:36.438534 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a63f5095-17f5-418f-8722-a7c450e0dbcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:36.439377 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54399d-e339-4ed8-b4b9-13a8e5649103 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:36.439668 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54399d-e339-4ed8-b4b9-13a8e5649103 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:36.439974 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54399d-e339-4ed8-b4b9-13a8e5649103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:36.440184 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1202/2412] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:33:36 2022] DELETE /identity/v3/projects/30772bbd4cb74cef96dc2e5b4c100491 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:36.445007 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd37c11-b957-4c21-8ca2-9fba19858f2b None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:36.445278 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd37c11-b957-4c21-8ca2-9fba19858f2b None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:36.445547 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd37c11-b957-4c21-8ca2-9fba19858f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:36.467087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-7cd37c11-b957-4c21-8ca2-9fba19858f2b None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:36.468318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-7cd37c11-b957-4c21-8ca2-9fba19858f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:36.473737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-9a54399d-e339-4ed8-b4b9-13a8e5649103 None None] MFA Rules not processed for user `c83dbb9f66ef4e8b9bf7c4cdca80863a`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:36.475473 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-9a54399d-e339-4ed8-b4b9-13a8e5649103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:36.518819 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:36.532541 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1203/2413] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:36.540288 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b0725-71c0-4db5-8fb2-89b3f857ac64 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:36.540589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b0725-71c0-4db5-8fb2-89b3f857ac64 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:36.540864 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b0725-71c0-4db5-8fb2-89b3f857ac64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:36.542561 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1211/2414] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:33:36.555015 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54399d-e339-4ed8-b4b9-13a8e5649103 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:36.556364 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-9a54399d-e339-4ed8-b4b9-13a8e5649103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:36.564138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-d09b0725-71c0-4db5-8fb2-89b3f857ac64 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:36.565421 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-d09b0725-71c0-4db5-8fb2-89b3f857ac64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:36.573579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1204/2415] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:36.577744 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b0725-71c0-4db5-8fb2-89b3f857ac64 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:36.587280 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b0725-71c0-4db5-8fb2-89b3f857ac64 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:36.592666 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f3c7e-b9dd-49b3-9b36-ed3a48a26c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:36.593257 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f3c7e-b9dd-49b3-9b36-ed3a48a26c71 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:36.593532 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f3c7e-b9dd-49b3-9b36-ed3a48a26c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:36.593798 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f3c7e-b9dd-49b3-9b36-ed3a48a26c71 admin admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:36.594434 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f3c7e-b9dd-49b3-9b36-ed3a48a26c71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:33:36.594732 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f3c7e-b9dd-49b3-9b36-ed3a48a26c71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:36.596357 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f3c7e-b9dd-49b3-9b36-ed3a48a26c71 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:36.605005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:36.609148 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1205/2416] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:33:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:33:36.613539 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f3c7e-b9dd-49b3-9b36-ed3a48a26c71 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:36.619041 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54399d-e339-4ed8-b4b9-13a8e5649103 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:36.623088 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a71206-7b94-4bb6-a918-746ace88e7ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:36.623702 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a71206-7b94-4bb6-a918-746ace88e7ff service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:36.623888 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f3c7e-b9dd-49b3-9b36-ed3a48a26c71 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:36.624012 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a71206-7b94-4bb6-a918-746ace88e7ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:36.624268 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a71206-7b94-4bb6-a918-746ace88e7ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:36.628568 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a403ad7-646b-4f85-8b03-03597b16d932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:36.629151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a403ad7-646b-4f85-8b03-03597b16d932 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:36.629420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a403ad7-646b-4f85-8b03-03597b16d932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:36.629701 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a403ad7-646b-4f85-8b03-03597b16d932 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:36.630309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a403ad7-646b-4f85-8b03-03597b16d932 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:36.632640 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a403ad7-646b-4f85-8b03-03597b16d932 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:36.633380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a71206-7b94-4bb6-a918-746ace88e7ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:36.634828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a71206-7b94-4bb6-a918-746ace88e7ff service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:36.644299 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1212/2417] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:33:36.663393 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1206/2418] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:33:36 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:33:36.664983 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a71206-7b94-4bb6-a918-746ace88e7ff service cinder] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:36.676110 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a71206-7b94-4bb6-a918-746ace88e7ff service cinder] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:36.733992 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d2f62-fbd7-4f2d-9d3c-8337791ebb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:36.734605 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d2f62-fbd7-4f2d-9d3c-8337791ebb92 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:36.734888 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d2f62-fbd7-4f2d-9d3c-8337791ebb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:36.735163 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d2f62-fbd7-4f2d-9d3c-8337791ebb92 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:36.735782 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d2f62-fbd7-4f2d-9d3c-8337791ebb92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:36.737780 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d2f62-fbd7-4f2d-9d3c-8337791ebb92 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:36.770069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1213/2419] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:33:36 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:33:36.771546 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a403ad7-646b-4f85-8b03-03597b16d932 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:36.782776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a403ad7-646b-4f85-8b03-03597b16d932 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:36.787753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1606c-c876-41ed-af7a-1abc28b8b0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:36.788938 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1606c-c876-41ed-af7a-1abc28b8b0d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:36.789254 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1606c-c876-41ed-af7a-1abc28b8b0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:36.789526 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1606c-c876-41ed-af7a-1abc28b8b0d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:36.790210 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b1606c-c876-41ed-af7a-1abc28b8b0d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:36.791863 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b1606c-c876-41ed-af7a-1abc28b8b0d1 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:36.802332 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1207/2420] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:36.807188 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d2f62-fbd7-4f2d-9d3c-8337791ebb92 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:36.818294 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d2f62-fbd7-4f2d-9d3c-8337791ebb92 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:36.823797 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2aa2c5-94d5-4d1a-ae55-d05996b977c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:36.824471 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2aa2c5-94d5-4d1a-ae55-d05996b977c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:36.824786 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2aa2c5-94d5-4d1a-ae55-d05996b977c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:36.825090 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2aa2c5-94d5-4d1a-ae55-d05996b977c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:36.825788 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2aa2c5-94d5-4d1a-ae55-d05996b977c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:36.827835 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2aa2c5-94d5-4d1a-ae55-d05996b977c7 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:36.839708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1214/2421] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:33:36.844111 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1606c-c876-41ed-af7a-1abc28b8b0d1 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:36.855580 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1606c-c876-41ed-af7a-1abc28b8b0d1 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:36.862015 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67736d-99e4-4a2b-8c68-68a904ec6963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:36.862687 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67736d-99e4-4a2b-8c68-68a904ec6963 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:36.862960 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67736d-99e4-4a2b-8c68-68a904ec6963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:36.863224 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67736d-99e4-4a2b-8c68-68a904ec6963 admin admin] PATH_INFO: `/v3/projects/0a45ac3e3b7b4338b99aa1d59cd5bbdf/users/03ef16f135434e5fa6e7e783d5793a12/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:36.879185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d67736d-99e4-4a2b-8c68-68a904ec6963 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a45ac3e3b7b4338b99aa1d59cd5bbdf, user_id=03ef16f135434e5fa6e7e783d5793a12, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:36.882657 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d67736d-99e4-4a2b-8c68-68a904ec6963 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:36.906434 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1208/2422] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:33:36 2022] PUT /identity/v3/projects/0a45ac3e3b7b4338b99aa1d59cd5bbdf/users/03ef16f135434e5fa6e7e783d5793a12/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:36.918162 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1cf6f-f77e-461d-b18d-903d27f97517 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:36.918620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1cf6f-f77e-461d-b18d-903d27f97517 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:36.918995 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1cf6f-f77e-461d-b18d-903d27f97517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:36.949728 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-b1b1cf6f-f77e-461d-b18d-903d27f97517 None None] MFA Rules not processed for user `03ef16f135434e5fa6e7e783d5793a12`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:36.950959 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-b1b1cf6f-f77e-461d-b18d-903d27f97517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:37.017633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1215/2423] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:33:37.022909 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67736d-99e4-4a2b-8c68-68a904ec6963 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:37.091614 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67736d-99e4-4a2b-8c68-68a904ec6963 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:37.097151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f893b63a-17a9-47dc-ab18-8e17f1084c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:37.097764 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893b63a-17a9-47dc-ab18-8e17f1084c33 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:37.098041 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893b63a-17a9-47dc-ab18-8e17f1084c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:37.098311 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893b63a-17a9-47dc-ab18-8e17f1084c33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:37.098934 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f893b63a-17a9-47dc-ab18-8e17f1084c33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:37.101274 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f893b63a-17a9-47dc-ab18-8e17f1084c33 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:37.133219 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1209/2424] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:33:37 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:37.134344 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1cf6f-f77e-461d-b18d-903d27f97517 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:37.145463 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1cf6f-f77e-461d-b18d-903d27f97517 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:37.201453 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:37.209733 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60dd75-2f56-4f5d-b422-d6fac82aad98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:37.210435 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60dd75-2f56-4f5d-b422-d6fac82aad98 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:37.210878 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60dd75-2f56-4f5d-b422-d6fac82aad98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:37.211221 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60dd75-2f56-4f5d-b422-d6fac82aad98 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:37.212083 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60dd75-2f56-4f5d-b422-d6fac82aad98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:37.214411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60dd75-2f56-4f5d-b422-d6fac82aad98 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:37.238080 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1216/2425] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:33:37 2022] POST /identity/v3/users => generated 299 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:37.242165 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f893b63a-17a9-47dc-ab18-8e17f1084c33 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:37.253968 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f893b63a-17a9-47dc-ab18-8e17f1084c33 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:37.258998 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02474814-b064-494b-801f-6614a87df72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:37.259613 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02474814-b064-494b-801f-6614a87df72a admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:37.259894 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02474814-b064-494b-801f-6614a87df72a admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:37.260207 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02474814-b064-494b-801f-6614a87df72a admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:37.260860 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02474814-b064-494b-801f-6614a87df72a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:37.262501 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02474814-b064-494b-801f-6614a87df72a admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:37.274648 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1210/2426] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:33:37.278815 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60dd75-2f56-4f5d-b422-d6fac82aad98 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:37.290687 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60dd75-2f56-4f5d-b422-d6fac82aad98 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:37.296633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc7b6e-48d9-457e-88c9-ef8b21ef54b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:37.297296 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc7b6e-48d9-457e-88c9-ef8b21ef54b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:37.297577 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc7b6e-48d9-457e-88c9-ef8b21ef54b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:37.297846 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc7b6e-48d9-457e-88c9-ef8b21ef54b4 admin admin] PATH_INFO: `/v3/projects/9b81c9d7687949cabff900ae56f8986c/users/785570c3da2544b38ad8350aed87119a/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:37.315758 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc7b6e-48d9-457e-88c9-ef8b21ef54b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b81c9d7687949cabff900ae56f8986c, user_id=785570c3da2544b38ad8350aed87119a, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:37.318466 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc7b6e-48d9-457e-88c9-ef8b21ef54b4 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:37.333844 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1217/2427] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:33:37 2022] PUT /identity/v3/projects/9b81c9d7687949cabff900ae56f8986c/users/785570c3da2544b38ad8350aed87119a/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:37.338023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02474814-b064-494b-801f-6614a87df72a admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:37.352199 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02474814-b064-494b-801f-6614a87df72a admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:37.414033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:37.426565 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf43db-f5ec-4f39-95e4-147705f39e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:37.427183 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf43db-f5ec-4f39-95e4-147705f39e5c admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:37.427462 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf43db-f5ec-4f39-95e4-147705f39e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:37.427736 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf43db-f5ec-4f39-95e4-147705f39e5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:37.428635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf43db-f5ec-4f39-95e4-147705f39e5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:37.430331 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf43db-f5ec-4f39-95e4-147705f39e5c admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:37.440939 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1211/2428] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:37.445283 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc7b6e-48d9-457e-88c9-ef8b21ef54b4 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:37.455673 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc7b6e-48d9-457e-88c9-ef8b21ef54b4 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:37.461580 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1583f4e9-9c49-4713-832d-18ff29ecd4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:37.462272 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583f4e9-9c49-4713-832d-18ff29ecd4b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:37.462546 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583f4e9-9c49-4713-832d-18ff29ecd4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:37.462813 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583f4e9-9c49-4713-832d-18ff29ecd4b9 admin admin] PATH_INFO: `/v3/projects/9b81c9d7687949cabff900ae56f8986c/users/785570c3da2544b38ad8350aed87119a/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:37.464823 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1583f4e9-9c49-4713-832d-18ff29ecd4b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b81c9d7687949cabff900ae56f8986c, user_id=785570c3da2544b38ad8350aed87119a, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:37.467473 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1583f4e9-9c49-4713-832d-18ff29ecd4b9 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:37.479002 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1218/2429] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:33:37 2022] PUT /identity/v3/projects/9b81c9d7687949cabff900ae56f8986c/users/785570c3da2544b38ad8350aed87119a/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:37.495181 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e408596-f96e-4c86-a5ec-441bd7d537bc None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:37.495467 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e408596-f96e-4c86-a5ec-441bd7d537bc None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:37.495793 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e408596-f96e-4c86-a5ec-441bd7d537bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:37.523631 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-9e408596-f96e-4c86-a5ec-441bd7d537bc None None] MFA Rules not processed for user `785570c3da2544b38ad8350aed87119a`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:37.524929 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-9e408596-f96e-4c86-a5ec-441bd7d537bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:37.592467 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1212/2430] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:37 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:37.603942 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1583f4e9-9c49-4713-832d-18ff29ecd4b9 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:37.605929 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-1583f4e9-9c49-4713-832d-18ff29ecd4b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:37.665781 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1583f4e9-9c49-4713-832d-18ff29ecd4b9 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:37.670821 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032412a-24ce-400e-b650-579801d1138b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:37.671465 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032412a-24ce-400e-b650-579801d1138b service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:37.671879 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032412a-24ce-400e-b650-579801d1138b service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:37.672322 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032412a-24ce-400e-b650-579801d1138b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:37.682271 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c032412a-24ce-400e-b650-579801d1138b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:37.684090 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c032412a-24ce-400e-b650-579801d1138b service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:37.693532 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1219/2431] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:37 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:33:37.841913 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e408596-f96e-4c86-a5ec-441bd7d537bc None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:37.907747 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e408596-f96e-4c86-a5ec-441bd7d537bc None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:37.913526 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdba654-d969-429d-8774-c76ed2b7f9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:37.914154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdba654-d969-429d-8774-c76ed2b7f9e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:37.914435 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdba654-d969-429d-8774-c76ed2b7f9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:37.914709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdba654-d969-429d-8774-c76ed2b7f9e6 admin admin] PATH_INFO: `/v3/users/03ef16f135434e5fa6e7e783d5793a12` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:37.915882 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdba654-d969-429d-8774-c76ed2b7f9e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03ef16f135434e5fa6e7e783d5793a12)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:37.917864 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdba654-d969-429d-8774-c76ed2b7f9e6 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:37.982681 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-cfdba654-d969-429d-8774-c76ed2b7f9e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03ef16f135434e5fa6e7e783d5793a12'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:37.991554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-cfdba654-d969-429d-8774-c76ed2b7f9e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03ef16f135434e5fa6e7e783d5793a12'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:38.007947 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-cfdba654-d969-429d-8774-c76ed2b7f9e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03ef16f135434e5fa6e7e783d5793a12'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:38.008674 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-cfdba654-d969-429d-8774-c76ed2b7f9e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03ef16f135434e5fa6e7e783d5793a12'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:38.011210 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:38.017770 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-cfdba654-d969-429d-8774-c76ed2b7f9e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03ef16f135434e5fa6e7e783d5793a12'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:38.025738 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1213/2432] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:33:37 2022] DELETE /identity/v3/users/03ef16f135434e5fa6e7e783d5793a12 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:38.030500 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7aff8-ebfa-4115-9abd-32a3d9c58c9b None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:38.030977 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7aff8-ebfa-4115-9abd-32a3d9c58c9b None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:38.030977 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7aff8-ebfa-4115-9abd-32a3d9c58c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:38.049879 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-6dd7aff8-ebfa-4115-9abd-32a3d9c58c9b None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:38.051082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-6dd7aff8-ebfa-4115-9abd-32a3d9c58c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:38.121331 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1220/2433] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:38.131472 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdba654-d969-429d-8774-c76ed2b7f9e6 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:38.132954 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-cfdba654-d969-429d-8774-c76ed2b7f9e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:38.200821 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdba654-d969-429d-8774-c76ed2b7f9e6 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:38.204555 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f136ae-bbbf-449a-8dc2-5f630f37f7ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:38.205160 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f136ae-bbbf-449a-8dc2-5f630f37f7ea service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:38.205433 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f136ae-bbbf-449a-8dc2-5f630f37f7ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:38.205698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f136ae-bbbf-449a-8dc2-5f630f37f7ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:38.214736 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f136ae-bbbf-449a-8dc2-5f630f37f7ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:38.216186 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f136ae-bbbf-449a-8dc2-5f630f37f7ea service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:38.225992 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1214/2434] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:33:38.520143 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd7aff8-ebfa-4115-9abd-32a3d9c58c9b None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:38.520143 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-6dd7aff8-ebfa-4115-9abd-32a3d9c58c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:38.530378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd7aff8-ebfa-4115-9abd-32a3d9c58c9b None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:38.535510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ec794-ff50-4073-b6fa-9809484c450c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:38.536179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ec794-ff50-4073-b6fa-9809484c450c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:38.536441 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ec794-ff50-4073-b6fa-9809484c450c admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:38.536709 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ec794-ff50-4073-b6fa-9809484c450c admin admin] PATH_INFO: `/v3/projects/0a45ac3e3b7b4338b99aa1d59cd5bbdf` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:38.537889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ec794-ff50-4073-b6fa-9809484c450c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a45ac3e3b7b4338b99aa1d59cd5bbdf)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:38.539796 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ec794-ff50-4073-b6fa-9809484c450c admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:38.586408 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:38.589392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-7d1ec794-ff50-4073-b6fa-9809484c450c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a45ac3e3b7b4338b99aa1d59cd5bbdf', 'request_id': 'req-7d1ec794-ff50-4073-b6fa-9809484c450c'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:38.596847 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-7d1ec794-ff50-4073-b6fa-9809484c450c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a45ac3e3b7b4338b99aa1d59cd5bbdf', 'request_id': 'req-7d1ec794-ff50-4073-b6fa-9809484c450c'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:38.606950 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-7d1ec794-ff50-4073-b6fa-9809484c450c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a45ac3e3b7b4338b99aa1d59cd5bbdf', 'request_id': 'req-7d1ec794-ff50-4073-b6fa-9809484c450c'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:38.618402 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-7d1ec794-ff50-4073-b6fa-9809484c450c admin admin] The token cache is being invalidate because project 0a45ac3e3b7b4338b99aa1d59cd5bbdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:33:38.618680 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-7d1ec794-ff50-4073-b6fa-9809484c450c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:38.619327 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-7d1ec794-ff50-4073-b6fa-9809484c450c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:38.620909 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1221/2435] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:33:38 2022] DELETE /identity/v3/projects/0a45ac3e3b7b4338b99aa1d59cd5bbdf => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:38.624966 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f136ae-bbbf-449a-8dc2-5f630f37f7ea service neutron] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:38.635515 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f136ae-bbbf-449a-8dc2-5f630f37f7ea service neutron] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:38.701945 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e86bd8-ffcd-4949-94f4-4d3a32afece0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:38.702661 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e86bd8-ffcd-4949-94f4-4d3a32afece0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:38.702978 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e86bd8-ffcd-4949-94f4-4d3a32afece0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:38.703308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e86bd8-ffcd-4949-94f4-4d3a32afece0 admin admin] PATH_INFO: `/v3/users/785570c3da2544b38ad8350aed87119a` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:38.704665 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e86bd8-ffcd-4949-94f4-4d3a32afece0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=785570c3da2544b38ad8350aed87119a)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:38.706855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e86bd8-ffcd-4949-94f4-4d3a32afece0 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:38.758112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-65e86bd8-ffcd-4949-94f4-4d3a32afece0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '785570c3da2544b38ad8350aed87119a'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:38.766912 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-65e86bd8-ffcd-4949-94f4-4d3a32afece0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '785570c3da2544b38ad8350aed87119a'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:38.780430 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-65e86bd8-ffcd-4949-94f4-4d3a32afece0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '785570c3da2544b38ad8350aed87119a'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:38.781086 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-65e86bd8-ffcd-4949-94f4-4d3a32afece0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '785570c3da2544b38ad8350aed87119a'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:38.788469 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-65e86bd8-ffcd-4949-94f4-4d3a32afece0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '785570c3da2544b38ad8350aed87119a'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:38.794526 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1215/2436] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:33:38 2022] DELETE /identity/v3/users/785570c3da2544b38ad8350aed87119a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:39.080859 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ec794-ff50-4073-b6fa-9809484c450c admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:39.082370 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-7d1ec794-ff50-4073-b6fa-9809484c450c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:39.098894 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ec794-ff50-4073-b6fa-9809484c450c admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:39.164079 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b579d-f8b5-49b9-91a6-6e2a914ed411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:39.164733 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b579d-f8b5-49b9-91a6-6e2a914ed411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:39.165000 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b579d-f8b5-49b9-91a6-6e2a914ed411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:39.165271 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b579d-f8b5-49b9-91a6-6e2a914ed411 admin admin] PATH_INFO: `/v3/projects/9b81c9d7687949cabff900ae56f8986c` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:39.166408 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b579d-f8b5-49b9-91a6-6e2a914ed411 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b81c9d7687949cabff900ae56f8986c)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:39.168372 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b579d-f8b5-49b9-91a6-6e2a914ed411 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:39.218482 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-079b579d-f8b5-49b9-91a6-6e2a914ed411 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b81c9d7687949cabff900ae56f8986c', 'request_id': 'req-079b579d-f8b5-49b9-91a6-6e2a914ed411'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:39.224826 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-079b579d-f8b5-49b9-91a6-6e2a914ed411 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b81c9d7687949cabff900ae56f8986c', 'request_id': 'req-079b579d-f8b5-49b9-91a6-6e2a914ed411'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:39.226342 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e86bd8-ffcd-4949-94f4-4d3a32afece0 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:39.228074 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-65e86bd8-ffcd-4949-94f4-4d3a32afece0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:39.233677 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-079b579d-f8b5-49b9-91a6-6e2a914ed411 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b81c9d7687949cabff900ae56f8986c', 'request_id': 'req-079b579d-f8b5-49b9-91a6-6e2a914ed411'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:39.242080 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e86bd8-ffcd-4949-94f4-4d3a32afece0 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:39.244701 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-079b579d-f8b5-49b9-91a6-6e2a914ed411 admin admin] The token cache is being invalidate because project 9b81c9d7687949cabff900ae56f8986c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:33:39.244990 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-079b579d-f8b5-49b9-91a6-6e2a914ed411 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:39.245627 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-079b579d-f8b5-49b9-91a6-6e2a914ed411 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:39.247266 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1222/2437] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:33:39 2022] DELETE /identity/v3/projects/9b81c9d7687949cabff900ae56f8986c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:39.251734 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab965c12-ac39-459e-87cb-354550277bae None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:39.252045 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab965c12-ac39-459e-87cb-354550277bae None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:39.252160 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:39.252321 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab965c12-ac39-459e-87cb-354550277bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:39.273419 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-ab965c12-ac39-459e-87cb-354550277bae None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:39.274893 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-ab965c12-ac39-459e-87cb-354550277bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:39.302888 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e67f1aa-c9b9-4326-93ea-b2baca5971f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:39.303651 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67f1aa-c9b9-4326-93ea-b2baca5971f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:39.303971 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67f1aa-c9b9-4326-93ea-b2baca5971f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:39.304241 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67f1aa-c9b9-4326-93ea-b2baca5971f3 admin admin] PATH_INFO: `/v3/users/c83dbb9f66ef4e8b9bf7c4cdca80863a` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:39.305480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e67f1aa-c9b9-4326-93ea-b2baca5971f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c83dbb9f66ef4e8b9bf7c4cdca80863a)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:39.307644 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e67f1aa-c9b9-4326-93ea-b2baca5971f3 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:39.336892 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1223/2438] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:39.345533 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e3abb-d756-45c9-9359-9611aa32ea24 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:39.345827 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e3abb-d756-45c9-9359-9611aa32ea24 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:39.346104 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e3abb-d756-45c9-9359-9611aa32ea24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:39.356338 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:39.358030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-6e67f1aa-c9b9-4326-93ea-b2baca5971f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c83dbb9f66ef4e8b9bf7c4cdca80863a'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:39.366596 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-6e67f1aa-c9b9-4326-93ea-b2baca5971f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c83dbb9f66ef4e8b9bf7c4cdca80863a'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:39.370137 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-6f1e3abb-d756-45c9-9359-9611aa32ea24 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:39.372825 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-6f1e3abb-d756-45c9-9359-9611aa32ea24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:39.380231 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-6e67f1aa-c9b9-4326-93ea-b2baca5971f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c83dbb9f66ef4e8b9bf7c4cdca80863a'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:39.380902 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-6e67f1aa-c9b9-4326-93ea-b2baca5971f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c83dbb9f66ef4e8b9bf7c4cdca80863a'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:39.383152 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1224/2439] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:39.387685 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e3abb-d756-45c9-9359-9611aa32ea24 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:39.391041 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-6e67f1aa-c9b9-4326-93ea-b2baca5971f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c83dbb9f66ef4e8b9bf7c4cdca80863a'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:39.397259 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1216/2440] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:33:39 2022] DELETE /identity/v3/users/c83dbb9f66ef4e8b9bf7c4cdca80863a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:33:39.402154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a568ca-0c8c-446c-b3cf-21386f6e2572 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:39.402411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e3abb-d756-45c9-9359-9611aa32ea24 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:39.402574 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a568ca-0c8c-446c-b3cf-21386f6e2572 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:39.402697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a568ca-0c8c-446c-b3cf-21386f6e2572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:39.422186 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-d0a568ca-0c8c-446c-b3cf-21386f6e2572 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:39.423240 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-d0a568ca-0c8c-446c-b3cf-21386f6e2572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:39.461969 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f559c12a-7a38-4d05-af82-f0dff328b714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:39.462592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f559c12a-7a38-4d05-af82-f0dff328b714 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:39.462874 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f559c12a-7a38-4d05-af82-f0dff328b714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:39.463143 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f559c12a-7a38-4d05-af82-f0dff328b714 admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:39.464065 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f559c12a-7a38-4d05-af82-f0dff328b714 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:33:39.464186 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f559c12a-7a38-4d05-af82-f0dff328b714 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:39.465752 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f559c12a-7a38-4d05-af82-f0dff328b714 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:39.482183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1225/2441] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:33:39.483151 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f559c12a-7a38-4d05-af82-f0dff328b714 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:39.484922 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-f559c12a-7a38-4d05-af82-f0dff328b714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:39.485644 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1217/2442] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:33:39.495824 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a568ca-0c8c-446c-b3cf-21386f6e2572 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:39.497208 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-d0a568ca-0c8c-446c-b3cf-21386f6e2572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:39.499247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f559c12a-7a38-4d05-af82-f0dff328b714 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:39.504188 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0f328d-3f75-4ead-8a67-0753ee647a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:39.504797 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f328d-3f75-4ead-8a67-0753ee647a56 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:39.505080 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f328d-3f75-4ead-8a67-0753ee647a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:39.505354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f328d-3f75-4ead-8a67-0753ee647a56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:39.505980 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0f328d-3f75-4ead-8a67-0753ee647a56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:39.508584 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0f328d-3f75-4ead-8a67-0753ee647a56 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:39.534270 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:39.542609 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1226/2443] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:33:39 2022] POST /identity/v3/projects => generated 379 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:39.543644 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0f328d-3f75-4ead-8a67-0753ee647a56 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:39.561471 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a568ca-0c8c-446c-b3cf-21386f6e2572 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:39.639183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0f328d-3f75-4ead-8a67-0753ee647a56 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:39.646486 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e39df4-6d1e-4e15-ade0-2680680e936c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:39.647460 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e39df4-6d1e-4e15-ade0-2680680e936c admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:39.647942 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e39df4-6d1e-4e15-ade0-2680680e936c admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:39.648393 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e39df4-6d1e-4e15-ade0-2680680e936c admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:39.649426 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e39df4-6d1e-4e15-ade0-2680680e936c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:39.650026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5cc486-22c6-4fe6-8763-f78818ed8125 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:39.650924 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5cc486-22c6-4fe6-8763-f78818ed8125 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:39.651390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5cc486-22c6-4fe6-8763-f78818ed8125 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:39.651821 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5cc486-22c6-4fe6-8763-f78818ed8125 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:39.652771 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e39df4-6d1e-4e15-ade0-2680680e936c admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:39.664068 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5cc486-22c6-4fe6-8763-f78818ed8125 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:39.665504 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5cc486-22c6-4fe6-8763-f78818ed8125 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:39.676681 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1218/2444] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:33:39.684648 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1227/2445] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:33:39 2022] POST /identity/v3/users => generated 290 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:39.685799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5cc486-22c6-4fe6-8763-f78818ed8125 service neutron] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:39.696313 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5cc486-22c6-4fe6-8763-f78818ed8125 service neutron] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:39.701196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619a9fa-cc94-4fe4-9b4a-93eedd7c3d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:39.701874 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619a9fa-cc94-4fe4-9b4a-93eedd7c3d56 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:39.702221 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619a9fa-cc94-4fe4-9b4a-93eedd7c3d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:39.702560 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619a9fa-cc94-4fe4-9b4a-93eedd7c3d56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:39.703271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8619a9fa-cc94-4fe4-9b4a-93eedd7c3d56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:39.705015 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8619a9fa-cc94-4fe4-9b4a-93eedd7c3d56 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:39.716339 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1219/2446] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:39.721216 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e39df4-6d1e-4e15-ade0-2680680e936c admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:39.734989 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e39df4-6d1e-4e15-ade0-2680680e936c admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:39.740726 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8826950-7188-4b29-b58f-cdc5af487e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:39.741939 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8826950-7188-4b29-b58f-cdc5af487e92 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:39.742154 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8826950-7188-4b29-b58f-cdc5af487e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:39.742507 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8826950-7188-4b29-b58f-cdc5af487e92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:39.743419 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8826950-7188-4b29-b58f-cdc5af487e92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:39.745187 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8826950-7188-4b29-b58f-cdc5af487e92 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:39.756340 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1228/2447] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:39.760804 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619a9fa-cc94-4fe4-9b4a-93eedd7c3d56 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:39.771070 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619a9fa-cc94-4fe4-9b4a-93eedd7c3d56 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:39.776806 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53055843-e7da-481e-aa8a-96f639c9a31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:39.777573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53055843-e7da-481e-aa8a-96f639c9a31b admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:39.777940 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53055843-e7da-481e-aa8a-96f639c9a31b admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:39.778153 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53055843-e7da-481e-aa8a-96f639c9a31b admin admin] PATH_INFO: `/v3/projects/ea4e83f5c87641538b91b5b08f520e64/users/8fe5039a3488421eb318e5389d795954/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:39.794833 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53055843-e7da-481e-aa8a-96f639c9a31b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea4e83f5c87641538b91b5b08f520e64, user_id=8fe5039a3488421eb318e5389d795954, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:39.797524 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53055843-e7da-481e-aa8a-96f639c9a31b admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:39.819309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1220/2448] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:33:39 2022] PUT /identity/v3/projects/ea4e83f5c87641538b91b5b08f520e64/users/8fe5039a3488421eb318e5389d795954/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:39.822302 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7bc099-b809-40e1-8cff-224e116f185b None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:39.822589 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7bc099-b809-40e1-8cff-224e116f185b None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:39.822865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7bc099-b809-40e1-8cff-224e116f185b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:39.860077 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-be7bc099-b809-40e1-8cff-224e116f185b None None] MFA Rules not processed for user `8fe5039a3488421eb318e5389d795954`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:39.861378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-be7bc099-b809-40e1-8cff-224e116f185b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:39.901978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:39.931679 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1229/2449] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:39.936106 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53055843-e7da-481e-aa8a-96f639c9a31b admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:39.957618 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7bc099-b809-40e1-8cff-224e116f185b None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:39.959155 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-be7bc099-b809-40e1-8cff-224e116f185b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:40.016090 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53055843-e7da-481e-aa8a-96f639c9a31b admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:40.020775 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087a62e-1b69-4b89-ac46-af43101054d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:40.021383 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087a62e-1b69-4b89-ac46-af43101054d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:40.021665 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087a62e-1b69-4b89-ac46-af43101054d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:40.021937 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087a62e-1b69-4b89-ac46-af43101054d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:40.023285 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5087a62e-1b69-4b89-ac46-af43101054d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:40.025239 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5087a62e-1b69-4b89-ac46-af43101054d6 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:40.032173 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7bc099-b809-40e1-8cff-224e116f185b None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:40.036758 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb863c-c99b-4c83-9b5f-102e71dc14e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:40.037388 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb863c-c99b-4c83-9b5f-102e71dc14e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:40.037698 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb863c-c99b-4c83-9b5f-102e71dc14e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:40.038201 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb863c-c99b-4c83-9b5f-102e71dc14e8 admin admin] PATH_INFO: `/v3/projects/74a9c3f0e6ff4d67aafb5901cea5a67b` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:40.039323 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb863c-c99b-4c83-9b5f-102e71dc14e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74a9c3f0e6ff4d67aafb5901cea5a67b)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:40.041252 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb863c-c99b-4c83-9b5f-102e71dc14e8 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:40.058465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1221/2450] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:33:39 2022] POST /identity/v3/projects => generated 377 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:40.059479 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087a62e-1b69-4b89-ac46-af43101054d6 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:40.090613 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8bbb863c-c99b-4c83-9b5f-102e71dc14e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74a9c3f0e6ff4d67aafb5901cea5a67b', 'request_id': 'req-8bbb863c-c99b-4c83-9b5f-102e71dc14e8'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:40.097322 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8bbb863c-c99b-4c83-9b5f-102e71dc14e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74a9c3f0e6ff4d67aafb5901cea5a67b', 'request_id': 'req-8bbb863c-c99b-4c83-9b5f-102e71dc14e8'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:40.106192 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8bbb863c-c99b-4c83-9b5f-102e71dc14e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74a9c3f0e6ff4d67aafb5901cea5a67b', 'request_id': 'req-8bbb863c-c99b-4c83-9b5f-102e71dc14e8'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:40.111124 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:40.118390 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8bbb863c-c99b-4c83-9b5f-102e71dc14e8 admin admin] The token cache is being invalidate because project 74a9c3f0e6ff4d67aafb5901cea5a67b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:33:40.118672 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8bbb863c-c99b-4c83-9b5f-102e71dc14e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:40.119335 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8bbb863c-c99b-4c83-9b5f-102e71dc14e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:40.120921 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1230/2451] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:33:39 2022] DELETE /identity/v3/projects/74a9c3f0e6ff4d67aafb5901cea5a67b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:33:40.130893 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087a62e-1b69-4b89-ac46-af43101054d6 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:40.195091 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1530d27-6a44-4033-984a-3d55403a31d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:40.196067 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1530d27-6a44-4033-984a-3d55403a31d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:40.196330 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1530d27-6a44-4033-984a-3d55403a31d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:40.196603 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1530d27-6a44-4033-984a-3d55403a31d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:40.197240 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1530d27-6a44-4033-984a-3d55403a31d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:40.199261 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1530d27-6a44-4033-984a-3d55403a31d4 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:40.222988 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1222/2452] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:33:40 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:33:40.228192 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb863c-c99b-4c83-9b5f-102e71dc14e8 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:40.229714 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-8bbb863c-c99b-4c83-9b5f-102e71dc14e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:40.243535 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb863c-c99b-4c83-9b5f-102e71dc14e8 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:40.248442 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e1738-fef7-493d-86b9-539a2ee354b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:40.249065 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e1738-fef7-493d-86b9-539a2ee354b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:40.249317 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e1738-fef7-493d-86b9-539a2ee354b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:40.249610 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e1738-fef7-493d-86b9-539a2ee354b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:40.250257 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668e1738-fef7-493d-86b9-539a2ee354b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:40.251868 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668e1738-fef7-493d-86b9-539a2ee354b9 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:40.267098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1231/2453] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:40.270949 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1530d27-6a44-4033-984a-3d55403a31d4 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:40.287236 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1530d27-6a44-4033-984a-3d55403a31d4 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:40.294263 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0d7f0-d71d-413d-9fae-1b465708ca3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:40.295031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0d7f0-d71d-413d-9fae-1b465708ca3c admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:40.295448 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0d7f0-d71d-413d-9fae-1b465708ca3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:40.295822 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0d7f0-d71d-413d-9fae-1b465708ca3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:40.296653 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f0d7f0-d71d-413d-9fae-1b465708ca3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:40.298812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f0d7f0-d71d-413d-9fae-1b465708ca3c admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:40.316858 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1223/2454] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:40.318245 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e1738-fef7-493d-86b9-539a2ee354b9 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:40.330125 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e1738-fef7-493d-86b9-539a2ee354b9 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:40.335763 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c646b-ae03-4410-aa88-2cbf278894b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:40.336653 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c646b-ae03-4410-aa88-2cbf278894b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:40.336999 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c646b-ae03-4410-aa88-2cbf278894b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:40.337333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c646b-ae03-4410-aa88-2cbf278894b4 admin admin] PATH_INFO: `/v3/projects/48d610dc090740c28e55be75211043c2/users/ba742d5c0d78400e94b4c476590f77c3/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:40.354422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c646b-ae03-4410-aa88-2cbf278894b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48d610dc090740c28e55be75211043c2, user_id=ba742d5c0d78400e94b4c476590f77c3, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:40.357191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c646b-ae03-4410-aa88-2cbf278894b4 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:40.372894 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1232/2455] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:33:40 2022] PUT /identity/v3/projects/48d610dc090740c28e55be75211043c2/users/ba742d5c0d78400e94b4c476590f77c3/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:33:40.383252 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b6e78-7577-4882-b1ed-eea0eed6019e None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:40.383540 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b6e78-7577-4882-b1ed-eea0eed6019e None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:40.383819 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b6e78-7577-4882-b1ed-eea0eed6019e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:40.414939 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-378b6e78-7577-4882-b1ed-eea0eed6019e None None] MFA Rules not processed for user `ba742d5c0d78400e94b4c476590f77c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:40.416177 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-378b6e78-7577-4882-b1ed-eea0eed6019e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:40.487158 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1224/2456] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:40.492086 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c646b-ae03-4410-aa88-2cbf278894b4 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:40.527295 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:40.563014 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c646b-ae03-4410-aa88-2cbf278894b4 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:40.568355 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4da2b3-1320-48c1-881f-0284c99f474e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:40.568975 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4da2b3-1320-48c1-881f-0284c99f474e admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:40.569261 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4da2b3-1320-48c1-881f-0284c99f474e admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:40.569540 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4da2b3-1320-48c1-881f-0284c99f474e admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:40.570156 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4da2b3-1320-48c1-881f-0284c99f474e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:40.572110 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4da2b3-1320-48c1-881f-0284c99f474e admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:40.601410 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1233/2457] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:33:40 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:40.606631 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b6e78-7577-4882-b1ed-eea0eed6019e None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:40.617677 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b6e78-7577-4882-b1ed-eea0eed6019e None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:40.670012 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:40.676952 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711cfd2-b55b-4eea-ab0b-40b5eb9fe854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:40.677737 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711cfd2-b55b-4eea-ab0b-40b5eb9fe854 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:40.678022 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711cfd2-b55b-4eea-ab0b-40b5eb9fe854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:40.678330 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711cfd2-b55b-4eea-ab0b-40b5eb9fe854 admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:40.679166 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711cfd2-b55b-4eea-ab0b-40b5eb9fe854 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:40.681165 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711cfd2-b55b-4eea-ab0b-40b5eb9fe854 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:40.705973 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1225/2458] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:33:40 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:40.710273 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4da2b3-1320-48c1-881f-0284c99f474e admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:40.723968 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4da2b3-1320-48c1-881f-0284c99f474e admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:40.729190 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241d186-955b-4269-bd1f-6ecb658123a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:40.729787 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241d186-955b-4269-bd1f-6ecb658123a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:40.730136 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241d186-955b-4269-bd1f-6ecb658123a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:40.730396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241d186-955b-4269-bd1f-6ecb658123a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:40.731066 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2241d186-955b-4269-bd1f-6ecb658123a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:40.732689 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2241d186-955b-4269-bd1f-6ecb658123a8 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:40.744270 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1234/2459] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:40.752818 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711cfd2-b55b-4eea-ab0b-40b5eb9fe854 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:40.768544 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711cfd2-b55b-4eea-ab0b-40b5eb9fe854 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:40.774505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928f152-30f5-4982-9204-1fcab82a2215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:40.775166 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928f152-30f5-4982-9204-1fcab82a2215 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:40.775438 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928f152-30f5-4982-9204-1fcab82a2215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:40.775707 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928f152-30f5-4982-9204-1fcab82a2215 admin admin] PATH_INFO: `/v3/projects/7fb45f90d2244614916c59327101db64/users/59ed7da965d1464fb31fad07af45df5a/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:40.792674 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928f152-30f5-4982-9204-1fcab82a2215 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fb45f90d2244614916c59327101db64, user_id=59ed7da965d1464fb31fad07af45df5a, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:40.795338 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928f152-30f5-4982-9204-1fcab82a2215 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:40.811754 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1226/2460] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:33:40 2022] PUT /identity/v3/projects/7fb45f90d2244614916c59327101db64/users/59ed7da965d1464fb31fad07af45df5a/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:40.815799 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241d186-955b-4269-bd1f-6ecb658123a8 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:40.826819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241d186-955b-4269-bd1f-6ecb658123a8 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:40.887793 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68d83c-27b6-440a-b2de-b88dd2fafe96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:40.888432 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68d83c-27b6-440a-b2de-b88dd2fafe96 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:40.888717 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68d83c-27b6-440a-b2de-b88dd2fafe96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:40.888989 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68d83c-27b6-440a-b2de-b88dd2fafe96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:40.889632 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68d83c-27b6-440a-b2de-b88dd2fafe96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:40.891233 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68d83c-27b6-440a-b2de-b88dd2fafe96 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:40.901504 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1235/2461] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:40.905918 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928f152-30f5-4982-9204-1fcab82a2215 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:40.917507 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928f152-30f5-4982-9204-1fcab82a2215 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:40.923058 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ef14c-fbde-48f2-8833-d57867605d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:40.923721 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ef14c-fbde-48f2-8833-d57867605d24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:40.924037 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ef14c-fbde-48f2-8833-d57867605d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:40.924312 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ef14c-fbde-48f2-8833-d57867605d24 admin admin] PATH_INFO: `/v3/projects/7fb45f90d2244614916c59327101db64/users/59ed7da965d1464fb31fad07af45df5a/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:40.926409 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ef14c-fbde-48f2-8833-d57867605d24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fb45f90d2244614916c59327101db64, user_id=59ed7da965d1464fb31fad07af45df5a, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:40.929618 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ef14c-fbde-48f2-8833-d57867605d24 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:40.939938 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1227/2462] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:33:40 2022] PUT /identity/v3/projects/7fb45f90d2244614916c59327101db64/users/59ed7da965d1464fb31fad07af45df5a/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:40.952782 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b36d17-fe07-4594-aea2-d584b7beec25 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:40.953154 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b36d17-fe07-4594-aea2-d584b7beec25 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:40.953526 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b36d17-fe07-4594-aea2-d584b7beec25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:40.992319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-b7b36d17-fe07-4594-aea2-d584b7beec25 None None] MFA Rules not processed for user `59ed7da965d1464fb31fad07af45df5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:40.993611 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-b7b36d17-fe07-4594-aea2-d584b7beec25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:41.064018 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1236/2463] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:40 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:41.077501 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ef14c-fbde-48f2-8833-d57867605d24 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:41.080090 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-605ef14c-fbde-48f2-8833-d57867605d24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:41.156197 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ef14c-fbde-48f2-8833-d57867605d24 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:41.161125 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c59ec-50d4-4343-9c72-4ec1018a7291 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:41.161799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c59ec-50d4-4343-9c72-4ec1018a7291 service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:41.162078 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c59ec-50d4-4343-9c72-4ec1018a7291 service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:41.162452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c59ec-50d4-4343-9c72-4ec1018a7291 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:41.174628 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c59ec-50d4-4343-9c72-4ec1018a7291 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:41.176136 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c59ec-50d4-4343-9c72-4ec1018a7291 service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:41.187038 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1228/2464] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:33:41.235904 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b36d17-fe07-4594-aea2-d584b7beec25 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:41.237512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-b7b36d17-fe07-4594-aea2-d584b7beec25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:41.251203 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b36d17-fe07-4594-aea2-d584b7beec25 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:41.255026 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea11bda-a7ec-4336-85b5-e5575fc30e2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:41.255666 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea11bda-a7ec-4336-85b5-e5575fc30e2e service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:41.255935 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea11bda-a7ec-4336-85b5-e5575fc30e2e service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:41.256238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea11bda-a7ec-4336-85b5-e5575fc30e2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:41.258570 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-9ea11bda-a7ec-4336-85b5-e5575fc30e2e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:41.286820 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:41.326091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea11bda-a7ec-4336-85b5-e5575fc30e2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:41.327569 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea11bda-a7ec-4336-85b5-e5575fc30e2e service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:41.337093 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1237/2465] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:33:44.324872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c59ec-50d4-4343-9c72-4ec1018a7291 service cinder] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:44.396622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c59ec-50d4-4343-9c72-4ec1018a7291 service cinder] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:44.402699 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e364acf-5b58-473d-8652-38805952b2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:44.403356 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e364acf-5b58-473d-8652-38805952b2b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:44.403634 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e364acf-5b58-473d-8652-38805952b2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:44.403929 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e364acf-5b58-473d-8652-38805952b2b2 admin admin] PATH_INFO: `/v3/users/8fe5039a3488421eb318e5389d795954` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:44.405095 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e364acf-5b58-473d-8652-38805952b2b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fe5039a3488421eb318e5389d795954)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:44.407054 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e364acf-5b58-473d-8652-38805952b2b2 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:44.464642 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9e364acf-5b58-473d-8652-38805952b2b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fe5039a3488421eb318e5389d795954'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:44.474943 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9e364acf-5b58-473d-8652-38805952b2b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fe5039a3488421eb318e5389d795954'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:44.476592 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=86,keystone:INSERT=6,keystone:DELETE=8 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:44.489681 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9e364acf-5b58-473d-8652-38805952b2b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fe5039a3488421eb318e5389d795954'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:44.490311 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9e364acf-5b58-473d-8652-38805952b2b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fe5039a3488421eb318e5389d795954'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:44.497513 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9e364acf-5b58-473d-8652-38805952b2b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fe5039a3488421eb318e5389d795954'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:44.503252 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1229/2466] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:33:44 2022] DELETE /identity/v3/users/8fe5039a3488421eb318e5389d795954 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:44.507823 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c1fb2-7db4-40a5-8d73-ba740c4547c2 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:44.508368 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c1fb2-7db4-40a5-8d73-ba740c4547c2 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:44.508546 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c1fb2-7db4-40a5-8d73-ba740c4547c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:44.528592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-1b9c1fb2-7db4-40a5-8d73-ba740c4547c2 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:44.529860 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-1b9c1fb2-7db4-40a5-8d73-ba740c4547c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:44.603122 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1238/2467] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:44.613965 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e364acf-5b58-473d-8652-38805952b2b2 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:44.615447 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-9e364acf-5b58-473d-8652-38805952b2b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:44.679449 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e364acf-5b58-473d-8652-38805952b2b2 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:44.683881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c9011-bfa9-4b1e-87b1-915b6f22f0a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:44.684566 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c9011-bfa9-4b1e-87b1-915b6f22f0a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:44.684847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c9011-bfa9-4b1e-87b1-915b6f22f0a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:44.685121 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c9011-bfa9-4b1e-87b1-915b6f22f0a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:44.695640 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c9011-bfa9-4b1e-87b1-915b6f22f0a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:44.697108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c9011-bfa9-4b1e-87b1-915b6f22f0a8 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:44.708530 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1230/2468] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:33:45.001569 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c1fb2-7db4-40a5-8d73-ba740c4547c2 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:45.003127 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-1b9c1fb2-7db4-40a5-8d73-ba740c4547c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:45.016500 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c1fb2-7db4-40a5-8d73-ba740c4547c2 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:45.021801 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb33a8-0986-4b23-a2fe-72e9d84df08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:45.022461 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb33a8-0986-4b23-a2fe-72e9d84df08b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:45.022741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb33a8-0986-4b23-a2fe-72e9d84df08b admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:45.023000 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb33a8-0986-4b23-a2fe-72e9d84df08b admin admin] PATH_INFO: `/v3/projects/ea4e83f5c87641538b91b5b08f520e64` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:45.024261 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb33a8-0986-4b23-a2fe-72e9d84df08b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea4e83f5c87641538b91b5b08f520e64)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:45.026241 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb33a8-0986-4b23-a2fe-72e9d84df08b admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:45.074926 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-3dfb33a8-0986-4b23-a2fe-72e9d84df08b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea4e83f5c87641538b91b5b08f520e64', 'request_id': 'req-3dfb33a8-0986-4b23-a2fe-72e9d84df08b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:45.081372 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-3dfb33a8-0986-4b23-a2fe-72e9d84df08b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea4e83f5c87641538b91b5b08f520e64', 'request_id': 'req-3dfb33a8-0986-4b23-a2fe-72e9d84df08b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:45.091326 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-3dfb33a8-0986-4b23-a2fe-72e9d84df08b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea4e83f5c87641538b91b5b08f520e64', 'request_id': 'req-3dfb33a8-0986-4b23-a2fe-72e9d84df08b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:45.102479 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-3dfb33a8-0986-4b23-a2fe-72e9d84df08b admin admin] The token cache is being invalidate because project ea4e83f5c87641538b91b5b08f520e64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:33:45.102792 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-3dfb33a8-0986-4b23-a2fe-72e9d84df08b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:45.103454 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-3dfb33a8-0986-4b23-a2fe-72e9d84df08b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:45.105073 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1239/2469] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:33:45 2022] DELETE /identity/v3/projects/ea4e83f5c87641538b91b5b08f520e64 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:45.110796 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c9011-bfa9-4b1e-87b1-915b6f22f0a8 service neutron] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:45.120958 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c9011-bfa9-4b1e-87b1-915b6f22f0a8 service neutron] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:45.178097 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e1910-7055-4d5a-abec-2678ddc32380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:45.178734 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e1910-7055-4d5a-abec-2678ddc32380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:45.179020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e1910-7055-4d5a-abec-2678ddc32380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:45.179291 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e1910-7055-4d5a-abec-2678ddc32380 admin admin] PATH_INFO: `/v3/users/ba742d5c0d78400e94b4c476590f77c3` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:45.180471 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e1910-7055-4d5a-abec-2678ddc32380 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba742d5c0d78400e94b4c476590f77c3)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:45.182626 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e1910-7055-4d5a-abec-2678ddc32380 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:45.240309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-c97e1910-7055-4d5a-abec-2678ddc32380 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba742d5c0d78400e94b4c476590f77c3'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:45.250453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-c97e1910-7055-4d5a-abec-2678ddc32380 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba742d5c0d78400e94b4c476590f77c3'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:45.262838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-c97e1910-7055-4d5a-abec-2678ddc32380 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba742d5c0d78400e94b4c476590f77c3'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:45.263489 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-c97e1910-7055-4d5a-abec-2678ddc32380 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba742d5c0d78400e94b4c476590f77c3'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:45.268184 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:45.272053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-c97e1910-7055-4d5a-abec-2678ddc32380 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba742d5c0d78400e94b4c476590f77c3'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:45.280732 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1231/2470] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:33:45 2022] DELETE /identity/v3/users/ba742d5c0d78400e94b4c476590f77c3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:45.593315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb33a8-0986-4b23-a2fe-72e9d84df08b admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:45.595904 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-3dfb33a8-0986-4b23-a2fe-72e9d84df08b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:45.615263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb33a8-0986-4b23-a2fe-72e9d84df08b admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:45.663839 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:45.678859 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42743ec7-6d31-400f-a32b-91c61f8f8984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:45.679498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42743ec7-6d31-400f-a32b-91c61f8f8984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:45.679771 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42743ec7-6d31-400f-a32b-91c61f8f8984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:45.680096 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42743ec7-6d31-400f-a32b-91c61f8f8984 admin admin] PATH_INFO: `/v3/projects/48d610dc090740c28e55be75211043c2` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:45.681224 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42743ec7-6d31-400f-a32b-91c61f8f8984 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48d610dc090740c28e55be75211043c2)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:45.683248 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42743ec7-6d31-400f-a32b-91c61f8f8984 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:45.733958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-42743ec7-6d31-400f-a32b-91c61f8f8984 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48d610dc090740c28e55be75211043c2', 'request_id': 'req-42743ec7-6d31-400f-a32b-91c61f8f8984'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:45.740542 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-42743ec7-6d31-400f-a32b-91c61f8f8984 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48d610dc090740c28e55be75211043c2', 'request_id': 'req-42743ec7-6d31-400f-a32b-91c61f8f8984'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:45.749285 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-42743ec7-6d31-400f-a32b-91c61f8f8984 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48d610dc090740c28e55be75211043c2', 'request_id': 'req-42743ec7-6d31-400f-a32b-91c61f8f8984'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:45.759455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-42743ec7-6d31-400f-a32b-91c61f8f8984 admin admin] The token cache is being invalidate because project 48d610dc090740c28e55be75211043c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:33:45.759740 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-42743ec7-6d31-400f-a32b-91c61f8f8984 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:45.760429 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-42743ec7-6d31-400f-a32b-91c61f8f8984 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:45.761966 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1240/2471] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:33:45 2022] DELETE /identity/v3/projects/48d610dc090740c28e55be75211043c2 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:45.766120 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e1910-7055-4d5a-abec-2678ddc32380 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:45.767697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-c97e1910-7055-4d5a-abec-2678ddc32380 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:45.784390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e1910-7055-4d5a-abec-2678ddc32380 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:45.848274 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e4063-a33b-4b8f-857e-0d8d7ea48c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:45.848922 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e4063-a33b-4b8f-857e-0d8d7ea48c8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:45.849194 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e4063-a33b-4b8f-857e-0d8d7ea48c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:45.849465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e4063-a33b-4b8f-857e-0d8d7ea48c8a admin admin] PATH_INFO: `/v3/users/59ed7da965d1464fb31fad07af45df5a` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:45.851001 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e4063-a33b-4b8f-857e-0d8d7ea48c8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=59ed7da965d1464fb31fad07af45df5a)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:45.853016 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e4063-a33b-4b8f-857e-0d8d7ea48c8a admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:45.901294 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-c61e4063-a33b-4b8f-857e-0d8d7ea48c8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59ed7da965d1464fb31fad07af45df5a'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:45.909865 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-c61e4063-a33b-4b8f-857e-0d8d7ea48c8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59ed7da965d1464fb31fad07af45df5a'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:45.929891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-c61e4063-a33b-4b8f-857e-0d8d7ea48c8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59ed7da965d1464fb31fad07af45df5a'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:45.930892 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-c61e4063-a33b-4b8f-857e-0d8d7ea48c8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59ed7da965d1464fb31fad07af45df5a'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:45.938865 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-c61e4063-a33b-4b8f-857e-0d8d7ea48c8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59ed7da965d1464fb31fad07af45df5a'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:45.944829 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1232/2472] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:33:45 2022] DELETE /identity/v3/users/59ed7da965d1464fb31fad07af45df5a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:46.233874 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42743ec7-6d31-400f-a32b-91c61f8f8984 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:46.235536 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-42743ec7-6d31-400f-a32b-91c61f8f8984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:46.249603 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42743ec7-6d31-400f-a32b-91c61f8f8984 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:46.310323 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c3db6-11bd-47cc-bb70-7b0c652816af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:46.310890 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c3db6-11bd-47cc-bb70-7b0c652816af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:46.311171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c3db6-11bd-47cc-bb70-7b0c652816af admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:46.311472 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c3db6-11bd-47cc-bb70-7b0c652816af admin admin] PATH_INFO: `/v3/projects/7fb45f90d2244614916c59327101db64` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:46.312637 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09c3db6-11bd-47cc-bb70-7b0c652816af admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fb45f90d2244614916c59327101db64)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:46.314569 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09c3db6-11bd-47cc-bb70-7b0c652816af admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:46.365339 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-f09c3db6-11bd-47cc-bb70-7b0c652816af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fb45f90d2244614916c59327101db64', 'request_id': 'req-f09c3db6-11bd-47cc-bb70-7b0c652816af'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:46.371697 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-f09c3db6-11bd-47cc-bb70-7b0c652816af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fb45f90d2244614916c59327101db64', 'request_id': 'req-f09c3db6-11bd-47cc-bb70-7b0c652816af'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:46.380947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-f09c3db6-11bd-47cc-bb70-7b0c652816af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fb45f90d2244614916c59327101db64', 'request_id': 'req-f09c3db6-11bd-47cc-bb70-7b0c652816af'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:46.393076 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-f09c3db6-11bd-47cc-bb70-7b0c652816af admin admin] The token cache is being invalidate because project 7fb45f90d2244614916c59327101db64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:33:46.393360 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-f09c3db6-11bd-47cc-bb70-7b0c652816af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:46.394016 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-f09c3db6-11bd-47cc-bb70-7b0c652816af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:46.395576 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1241/2473] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:33:46 2022] DELETE /identity/v3/projects/7fb45f90d2244614916c59327101db64 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:46.402081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07c62b-a6dd-478e-a865-d25e777d2acd None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:46.402359 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07c62b-a6dd-478e-a865-d25e777d2acd None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:46.402624 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07c62b-a6dd-478e-a865-d25e777d2acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:46.422687 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-5a07c62b-a6dd-478e-a865-d25e777d2acd None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:46.423931 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-5a07c62b-a6dd-478e-a865-d25e777d2acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:46.485867 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1233/2474] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:46.494335 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2db84-9108-43e1-b8e8-0bd97b7a7332 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:46.494733 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2db84-9108-43e1-b8e8-0bd97b7a7332 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:46.494939 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2db84-9108-43e1-b8e8-0bd97b7a7332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:46.524788 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-67e2db84-9108-43e1-b8e8-0bd97b7a7332 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:46.526479 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-67e2db84-9108-43e1-b8e8-0bd97b7a7332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:46.535791 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1242/2475] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:46.540269 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a07c62b-a6dd-478e-a865-d25e777d2acd None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:46.550429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a07c62b-a6dd-478e-a865-d25e777d2acd None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:46.556329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dec63a-c547-491f-8b85-351208bcd026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:46.556933 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dec63a-c547-491f-8b85-351208bcd026 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:46.557203 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dec63a-c547-491f-8b85-351208bcd026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:46.557653 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dec63a-c547-491f-8b85-351208bcd026 admin admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:46.558295 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dec63a-c547-491f-8b85-351208bcd026 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:33:46.558596 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dec63a-c547-491f-8b85-351208bcd026 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:46.560226 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dec63a-c547-491f-8b85-351208bcd026 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:46.569418 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:46.573672 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1234/2476] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:33:46.578247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2db84-9108-43e1-b8e8-0bd97b7a7332 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:46.584009 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:46.588189 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2db84-9108-43e1-b8e8-0bd97b7a7332 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:46.594633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d5d51-65fb-44c9-88dd-3f222c4b393a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:46.595390 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d5d51-65fb-44c9-88dd-3f222c4b393a admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:46.595840 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d5d51-65fb-44c9-88dd-3f222c4b393a admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:46.596182 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d5d51-65fb-44c9-88dd-3f222c4b393a admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:46.596958 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86d5d51-65fb-44c9-88dd-3f222c4b393a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:46.598847 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86d5d51-65fb-44c9-88dd-3f222c4b393a admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:46.629016 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1243/2477] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:33:46 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:46.635296 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dec63a-c547-491f-8b85-351208bcd026 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:46.651982 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dec63a-c547-491f-8b85-351208bcd026 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:46.714871 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d786ae6-8063-4dd2-84ba-8186cb860dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:46.715892 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d786ae6-8063-4dd2-84ba-8186cb860dab admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:46.715892 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d786ae6-8063-4dd2-84ba-8186cb860dab admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:46.716178 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d786ae6-8063-4dd2-84ba-8186cb860dab admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:46.716643 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d786ae6-8063-4dd2-84ba-8186cb860dab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:46.718570 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d786ae6-8063-4dd2-84ba-8186cb860dab admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:46.743185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1235/2478] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:33:46 2022] POST /identity/v3/users => generated 286 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:46.747956 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d5d51-65fb-44c9-88dd-3f222c4b393a admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:46.758241 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d5d51-65fb-44c9-88dd-3f222c4b393a admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:46.763369 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ab8d5-aee5-41c9-95b7-40b188c31fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:46.764014 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ab8d5-aee5-41c9-95b7-40b188c31fa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:46.764294 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ab8d5-aee5-41c9-95b7-40b188c31fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:46.764562 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ab8d5-aee5-41c9-95b7-40b188c31fa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:46.765240 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1ab8d5-aee5-41c9-95b7-40b188c31fa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:46.766966 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1ab8d5-aee5-41c9-95b7-40b188c31fa9 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:46.778624 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1244/2479] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:46.782762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d786ae6-8063-4dd2-84ba-8186cb860dab admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:46.793143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d786ae6-8063-4dd2-84ba-8186cb860dab admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:46.799254 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6433b1b8-8556-4f5c-868f-64759132db7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:46.799861 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433b1b8-8556-4f5c-868f-64759132db7e admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:46.800194 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433b1b8-8556-4f5c-868f-64759132db7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:46.800471 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433b1b8-8556-4f5c-868f-64759132db7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:46.801156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6433b1b8-8556-4f5c-868f-64759132db7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:46.802772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6433b1b8-8556-4f5c-868f-64759132db7e admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:46.814178 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1236/2480] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:33:46.818216 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ab8d5-aee5-41c9-95b7-40b188c31fa9 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:46.828615 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ab8d5-aee5-41c9-95b7-40b188c31fa9 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:46.833552 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1916aec-e470-4732-961e-bd1df83c63f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:46.834372 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1916aec-e470-4732-961e-bd1df83c63f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:46.834730 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1916aec-e470-4732-961e-bd1df83c63f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:46.835008 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1916aec-e470-4732-961e-bd1df83c63f1 admin admin] PATH_INFO: `/v3/projects/16e0c2b8e2a8400b90fc58de9b69a5b1/users/59083982a8794b69a93e6944149ac65a/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:46.852083 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1916aec-e470-4732-961e-bd1df83c63f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16e0c2b8e2a8400b90fc58de9b69a5b1, user_id=59083982a8794b69a93e6944149ac65a, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:46.855096 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1916aec-e470-4732-961e-bd1df83c63f1 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:46.869281 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1245/2481] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:33:46 2022] PUT /identity/v3/projects/16e0c2b8e2a8400b90fc58de9b69a5b1/users/59083982a8794b69a93e6944149ac65a/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:46.877824 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dcb7f4-9c22-450c-8e26-40d5ac3ead98 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:46.878217 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dcb7f4-9c22-450c-8e26-40d5ac3ead98 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:46.878554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dcb7f4-9c22-450c-8e26-40d5ac3ead98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:46.898318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-e3dcb7f4-9c22-450c-8e26-40d5ac3ead98 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:46.899578 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-e3dcb7f4-9c22-450c-8e26-40d5ac3ead98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:46.959699 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1237/2482] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:46.969697 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1916aec-e470-4732-961e-bd1df83c63f1 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:46.971703 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-f1916aec-e470-4732-961e-bd1df83c63f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:47.031131 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1916aec-e470-4732-961e-bd1df83c63f1 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:47.035121 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b94d8a-f416-4152-9342-268f1867c7df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:47.035722 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b94d8a-f416-4152-9342-268f1867c7df service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:47.036029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b94d8a-f416-4152-9342-268f1867c7df service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:47.036322 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b94d8a-f416-4152-9342-268f1867c7df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:47.046404 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b94d8a-f416-4152-9342-268f1867c7df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:47.047840 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b94d8a-f416-4152-9342-268f1867c7df service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:47.058152 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1246/2483] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:33:48.379618 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b8ebc-80a6-492e-b425-b015989c0aa6 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:48.380114 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b8ebc-80a6-492e-b425-b015989c0aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:48.380197 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b8ebc-80a6-492e-b425-b015989c0aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:48.425666 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-fc1b8ebc-80a6-492e-b425-b015989c0aa6 None None] MFA Rules not processed for user `59083982a8794b69a93e6944149ac65a`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:48.428718 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-fc1b8ebc-80a6-492e-b425-b015989c0aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:48.573558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1238/2484] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:48.577874 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b94d8a-f416-4152-9342-268f1867c7df service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:48.590601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b94d8a-f416-4152-9342-268f1867c7df service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:48.596212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1319f-0668-442f-8414-1148d4a844b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:48.597135 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1319f-0668-442f-8414-1148d4a844b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:48.597644 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1319f-0668-442f-8414-1148d4a844b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:48.597985 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1319f-0668-442f-8414-1148d4a844b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:48.598831 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d1319f-0668-442f-8414-1148d4a844b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:48.601334 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d1319f-0668-442f-8414-1148d4a844b1 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:48.632589 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1247/2485] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:33:48 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:48.636751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b8ebc-80a6-492e-b425-b015989c0aa6 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:48.649411 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b8ebc-80a6-492e-b425-b015989c0aa6 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:48.656040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:48.716047 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3925a1-d60b-4557-ba1f-4b2e12000902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:48.716672 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3925a1-d60b-4557-ba1f-4b2e12000902 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:48.716955 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3925a1-d60b-4557-ba1f-4b2e12000902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:48.717231 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3925a1-d60b-4557-ba1f-4b2e12000902 admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:48.717862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3925a1-d60b-4557-ba1f-4b2e12000902 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:48.719814 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3925a1-d60b-4557-ba1f-4b2e12000902 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:48.741822 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1239/2486] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:33:48 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:48.746065 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1319f-0668-442f-8414-1148d4a844b1 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:48.756819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1319f-0668-442f-8414-1148d4a844b1 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:48.762011 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f59f61-5e0e-4fe4-b641-781bf2df26df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:48.762686 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f59f61-5e0e-4fe4-b641-781bf2df26df admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:48.763081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f59f61-5e0e-4fe4-b641-781bf2df26df admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:48.763417 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f59f61-5e0e-4fe4-b641-781bf2df26df admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:48.764188 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f59f61-5e0e-4fe4-b641-781bf2df26df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:48.765881 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f59f61-5e0e-4fe4-b641-781bf2df26df admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:48.777279 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1248/2487] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:48.781395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3925a1-d60b-4557-ba1f-4b2e12000902 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:48.792754 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3925a1-d60b-4557-ba1f-4b2e12000902 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:48.798415 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18be7a-68a0-4a6d-a172-8b0536903654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:48.799204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18be7a-68a0-4a6d-a172-8b0536903654 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:48.799572 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18be7a-68a0-4a6d-a172-8b0536903654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:48.799998 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18be7a-68a0-4a6d-a172-8b0536903654 admin admin] PATH_INFO: `/v3/projects/fd86d315464043dd8767ff1bfd7cd3d1/users/7a9994c849634a848073ac56e60b96f5/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:48.833647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d18be7a-68a0-4a6d-a172-8b0536903654 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd86d315464043dd8767ff1bfd7cd3d1, user_id=7a9994c849634a848073ac56e60b96f5, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:48.838126 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d18be7a-68a0-4a6d-a172-8b0536903654 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:48.854781 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1240/2488] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:33:48 2022] PUT /identity/v3/projects/fd86d315464043dd8767ff1bfd7cd3d1/users/7a9994c849634a848073ac56e60b96f5/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:48.858942 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f59f61-5e0e-4fe4-b641-781bf2df26df admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:48.869231 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f59f61-5e0e-4fe4-b641-781bf2df26df admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:48.933028 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707976cf-ca7d-4e8b-8f27-84fe408a2f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:48.933771 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707976cf-ca7d-4e8b-8f27-84fe408a2f4d admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:48.934057 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707976cf-ca7d-4e8b-8f27-84fe408a2f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:48.934326 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707976cf-ca7d-4e8b-8f27-84fe408a2f4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:48.934967 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707976cf-ca7d-4e8b-8f27-84fe408a2f4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:48.936621 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707976cf-ca7d-4e8b-8f27-84fe408a2f4d admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:48.947866 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1249/2489] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:48.951773 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18be7a-68a0-4a6d-a172-8b0536903654 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:48.965015 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18be7a-68a0-4a6d-a172-8b0536903654 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:48.969928 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab59638-3d85-4fd5-a636-d99fd356587e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:48.970625 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab59638-3d85-4fd5-a636-d99fd356587e admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:48.970903 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab59638-3d85-4fd5-a636-d99fd356587e admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:48.971179 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab59638-3d85-4fd5-a636-d99fd356587e admin admin] PATH_INFO: `/v3/projects/fd86d315464043dd8767ff1bfd7cd3d1/users/7a9994c849634a848073ac56e60b96f5/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:48.973125 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab59638-3d85-4fd5-a636-d99fd356587e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd86d315464043dd8767ff1bfd7cd3d1, user_id=7a9994c849634a848073ac56e60b96f5, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:48.976126 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab59638-3d85-4fd5-a636-d99fd356587e admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:48.988444 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1241/2490] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:33:48 2022] PUT /identity/v3/projects/fd86d315464043dd8767ff1bfd7cd3d1/users/7a9994c849634a848073ac56e60b96f5/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:49.710276 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1eded5-d3ce-420d-8b4d-4371063fa241 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:49.710815 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1eded5-d3ce-420d-8b4d-4371063fa241 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:49.711083 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1eded5-d3ce-420d-8b4d-4371063fa241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:49.748248 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:49.753205 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-6c1eded5-d3ce-420d-8b4d-4371063fa241 None None] MFA Rules not processed for user `7a9994c849634a848073ac56e60b96f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:49.755489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-6c1eded5-d3ce-420d-8b4d-4371063fa241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:49.845709 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1250/2491] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:49 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:49.863261 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab59638-3d85-4fd5-a636-d99fd356587e admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:49.865989 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-8ab59638-3d85-4fd5-a636-d99fd356587e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:50.065013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab59638-3d85-4fd5-a636-d99fd356587e admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:50.068661 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d37de-1155-4e4d-bd3c-cd4f6fec5dfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:50.069269 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d37de-1155-4e4d-bd3c-cd4f6fec5dfa service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:50.069545 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d37de-1155-4e4d-bd3c-cd4f6fec5dfa service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:50.069815 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d37de-1155-4e4d-bd3c-cd4f6fec5dfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:50.137388 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d37de-1155-4e4d-bd3c-cd4f6fec5dfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:50.138947 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d37de-1155-4e4d-bd3c-cd4f6fec5dfa service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:50.148437 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1242/2492] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:33:51.520813 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1eded5-d3ce-420d-8b4d-4371063fa241 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:51.533602 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1eded5-d3ce-420d-8b4d-4371063fa241 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:51.538016 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec88879-ff71-4898-a9e4-91f91772e01b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:51.538624 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec88879-ff71-4898-a9e4-91f91772e01b service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:51.538895 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec88879-ff71-4898-a9e4-91f91772e01b service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:51.539159 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec88879-ff71-4898-a9e4-91f91772e01b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:51.548837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec88879-ff71-4898-a9e4-91f91772e01b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:51.550258 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec88879-ff71-4898-a9e4-91f91772e01b service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:51.559562 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1251/2493] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:51 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:33:53.692583 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d37de-1155-4e4d-bd3c-cd4f6fec5dfa service cinder] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:53.712247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:53.751514 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d37de-1155-4e4d-bd3c-cd4f6fec5dfa service cinder] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:53.755463 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f32056-aaeb-470d-8dfa-779f10cd7b82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:53.756094 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f32056-aaeb-470d-8dfa-779f10cd7b82 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:53.756368 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f32056-aaeb-470d-8dfa-779f10cd7b82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:53.756632 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f32056-aaeb-470d-8dfa-779f10cd7b82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:53.758661 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-12f32056-aaeb-470d-8dfa-779f10cd7b82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:53.824516 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f32056-aaeb-470d-8dfa-779f10cd7b82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:53.825955 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f32056-aaeb-470d-8dfa-779f10cd7b82 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:53.836324 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1243/2494] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:33:57.115665 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec88879-ff71-4898-a9e4-91f91772e01b service cinder] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:57.117265 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-bec88879-ff71-4898-a9e4-91f91772e01b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:57.190688 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f32056-aaeb-470d-8dfa-779f10cd7b82 service neutron] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:57.191268 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec88879-ff71-4898-a9e4-91f91772e01b service cinder] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:57.192534 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-12f32056-aaeb-470d-8dfa-779f10cd7b82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:57.196591 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b47d78-eb8b-4e7d-ac29-9cd3a24fad94 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd503fe3cf6d428e962c14ab864ee608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:57.197066 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b47d78-eb8b-4e7d-ac29-9cd3a24fad94 service placement] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:57.197340 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b47d78-eb8b-4e7d-ac29-9cd3a24fad94 service placement] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:57.197743 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b47d78-eb8b-4e7d-ac29-9cd3a24fad94 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:57.199773 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-d7b47d78-eb8b-4e7d-ac29-9cd3a24fad94 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:57.212366 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f32056-aaeb-470d-8dfa-779f10cd7b82 service neutron] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:57.212521 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b47d78-eb8b-4e7d-ac29-9cd3a24fad94 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:57.214378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b47d78-eb8b-4e7d-ac29-9cd3a24fad94 service placement] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:57.217715 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34dc365-d063-44a8-979e-7f41cc9cff30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:57.218395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34dc365-d063-44a8-979e-7f41cc9cff30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:57.218679 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34dc365-d063-44a8-979e-7f41cc9cff30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:57.219024 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34dc365-d063-44a8-979e-7f41cc9cff30 admin admin] PATH_INFO: `/v3/users/a91872508ecd49d98bd95651412f85a2` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:57.220419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34dc365-d063-44a8-979e-7f41cc9cff30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a91872508ecd49d98bd95651412f85a2)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:57.222593 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34dc365-d063-44a8-979e-7f41cc9cff30 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:57.227299 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1252/2495] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:33:57.280672 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f34dc365-d063-44a8-979e-7f41cc9cff30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a91872508ecd49d98bd95651412f85a2'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:57.288767 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f34dc365-d063-44a8-979e-7f41cc9cff30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a91872508ecd49d98bd95651412f85a2'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:57.300716 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f34dc365-d063-44a8-979e-7f41cc9cff30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a91872508ecd49d98bd95651412f85a2'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:57.301356 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f34dc365-d063-44a8-979e-7f41cc9cff30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a91872508ecd49d98bd95651412f85a2'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:57.308221 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f34dc365-d063-44a8-979e-7f41cc9cff30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a91872508ecd49d98bd95651412f85a2'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:57.314095 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1244/2496] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:33:57 2022] DELETE /identity/v3/users/a91872508ecd49d98bd95651412f85a2 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:33:57.499611 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b47d78-eb8b-4e7d-ac29-9cd3a24fad94 service placement] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:57.511042 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b47d78-eb8b-4e7d-ac29-9cd3a24fad94 service placement] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:57.580796 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748f7f0-52ae-47a9-8e4f-2d2945ebe145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:57.581449 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748f7f0-52ae-47a9-8e4f-2d2945ebe145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:57.581720 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748f7f0-52ae-47a9-8e4f-2d2945ebe145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:57.581989 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748f7f0-52ae-47a9-8e4f-2d2945ebe145 admin admin] PATH_INFO: `/v3/projects/52ab946e7d0d4a5397945bfcbf49e723` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:57.583126 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748f7f0-52ae-47a9-8e4f-2d2945ebe145 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52ab946e7d0d4a5397945bfcbf49e723)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:57.585251 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748f7f0-52ae-47a9-8e4f-2d2945ebe145 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:57.602176 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:57.634517 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c748f7f0-52ae-47a9-8e4f-2d2945ebe145 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52ab946e7d0d4a5397945bfcbf49e723', 'request_id': 'req-c748f7f0-52ae-47a9-8e4f-2d2945ebe145'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:57.640956 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c748f7f0-52ae-47a9-8e4f-2d2945ebe145 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52ab946e7d0d4a5397945bfcbf49e723', 'request_id': 'req-c748f7f0-52ae-47a9-8e4f-2d2945ebe145'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:57.649121 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c748f7f0-52ae-47a9-8e4f-2d2945ebe145 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52ab946e7d0d4a5397945bfcbf49e723', 'request_id': 'req-c748f7f0-52ae-47a9-8e4f-2d2945ebe145'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:57.660153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c748f7f0-52ae-47a9-8e4f-2d2945ebe145 admin admin] The token cache is being invalidate because project 52ab946e7d0d4a5397945bfcbf49e723 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:33:57.660447 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c748f7f0-52ae-47a9-8e4f-2d2945ebe145 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:57.661075 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c748f7f0-52ae-47a9-8e4f-2d2945ebe145 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:57.662647 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1253/2497] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:33:57 2022] DELETE /identity/v3/projects/52ab946e7d0d4a5397945bfcbf49e723 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:57.667789 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbe795-8e8e-4be4-adcf-7e0a6c9ca7b4 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:57.668135 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbe795-8e8e-4be4-adcf-7e0a6c9ca7b4 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:57.668417 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbe795-8e8e-4be4-adcf-7e0a6c9ca7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:57.689011 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-69bbe795-8e8e-4be4-adcf-7e0a6c9ca7b4 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:57.690204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-69bbe795-8e8e-4be4-adcf-7e0a6c9ca7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:57.726446 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:57.756253 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1245/2498] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:57.765488 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3a211-3fbc-4b3f-8060-1c55be2f079e None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:57.765775 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3a211-3fbc-4b3f-8060-1c55be2f079e None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:57.766082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3a211-3fbc-4b3f-8060-1c55be2f079e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:57.789123 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-8fc3a211-3fbc-4b3f-8060-1c55be2f079e None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:57.790342 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-8fc3a211-3fbc-4b3f-8060-1c55be2f079e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:57.800223 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1254/2499] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:57.806244 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bbe795-8e8e-4be4-adcf-7e0a6c9ca7b4 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:57.816973 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bbe795-8e8e-4be4-adcf-7e0a6c9ca7b4 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:57.823615 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e922b3-5250-4348-866a-c0482bd1e85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:57.824268 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e922b3-5250-4348-866a-c0482bd1e85b admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:57.824545 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e922b3-5250-4348-866a-c0482bd1e85b admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:57.824816 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e922b3-5250-4348-866a-c0482bd1e85b admin admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:57.825467 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e922b3-5250-4348-866a-c0482bd1e85b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:33:57.825772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e922b3-5250-4348-866a-c0482bd1e85b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:57.827413 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e922b3-5250-4348-866a-c0482bd1e85b admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:57.840461 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1246/2500] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:33:57.844845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc3a211-3fbc-4b3f-8060-1c55be2f079e None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:57.856398 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc3a211-3fbc-4b3f-8060-1c55be2f079e None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:57.862140 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e3c0af-1df7-4301-b601-911ead29e326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:57.862740 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3c0af-1df7-4301-b601-911ead29e326 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:57.863038 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3c0af-1df7-4301-b601-911ead29e326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:57.863294 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3c0af-1df7-4301-b601-911ead29e326 admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:57.863947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e3c0af-1df7-4301-b601-911ead29e326 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:57.865832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e3c0af-1df7-4301-b601-911ead29e326 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:57.894250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1255/2501] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:33:57 2022] POST /identity/v3/projects => generated 403 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:57.898671 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e922b3-5250-4348-866a-c0482bd1e85b admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:57.909167 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e922b3-5250-4348-866a-c0482bd1e85b admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:57.967874 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3e0a9-cb10-43ee-b4c5-63338ec387e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:57.968432 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3e0a9-cb10-43ee-b4c5-63338ec387e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:57.968712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3e0a9-cb10-43ee-b4c5-63338ec387e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:57.968980 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3e0a9-cb10-43ee-b4c5-63338ec387e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:57.969668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec3e0a9-cb10-43ee-b4c5-63338ec387e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:57.971642 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec3e0a9-cb10-43ee-b4c5-63338ec387e9 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:57.997796 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1247/2502] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:33:57 2022] POST /identity/v3/users => generated 302 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:58.002061 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e3c0af-1df7-4301-b601-911ead29e326 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:58.015887 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e3c0af-1df7-4301-b601-911ead29e326 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:58.022234 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb3837-c3f2-4cd1-86cf-54254fb295b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:58.022843 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb3837-c3f2-4cd1-86cf-54254fb295b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:58.023124 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb3837-c3f2-4cd1-86cf-54254fb295b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:58.023396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb3837-c3f2-4cd1-86cf-54254fb295b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:58.024083 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eb3837-c3f2-4cd1-86cf-54254fb295b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:58.025682 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eb3837-c3f2-4cd1-86cf-54254fb295b6 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:58.036862 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1256/2503] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:58.041193 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3e0a9-cb10-43ee-b4c5-63338ec387e9 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:58.052106 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3e0a9-cb10-43ee-b4c5-63338ec387e9 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:58.057165 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d6c06-3a59-45d6-ad32-50698c747d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:58.057871 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d6c06-3a59-45d6-ad32-50698c747d0c admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:58.058143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d6c06-3a59-45d6-ad32-50698c747d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:58.058413 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d6c06-3a59-45d6-ad32-50698c747d0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:58.059108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d6c06-3a59-45d6-ad32-50698c747d0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:58.060851 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d6c06-3a59-45d6-ad32-50698c747d0c admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:58.072766 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1248/2504] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:33:58.077454 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb3837-c3f2-4cd1-86cf-54254fb295b6 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:58.092666 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb3837-c3f2-4cd1-86cf-54254fb295b6 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:58.099778 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e106cc0-75ea-4025-85f3-609b68d81833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:58.101083 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e106cc0-75ea-4025-85f3-609b68d81833 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:58.101473 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e106cc0-75ea-4025-85f3-609b68d81833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:58.101819 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e106cc0-75ea-4025-85f3-609b68d81833 admin admin] PATH_INFO: `/v3/projects/35b2abd6b6c846b7bada960950bde2bd/users/765b58262f9743459f9d9943a30ef22a/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:58.118963 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e106cc0-75ea-4025-85f3-609b68d81833 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35b2abd6b6c846b7bada960950bde2bd, user_id=765b58262f9743459f9d9943a30ef22a, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:58.121969 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e106cc0-75ea-4025-85f3-609b68d81833 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:58.138066 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1257/2505] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:33:58 2022] PUT /identity/v3/projects/35b2abd6b6c846b7bada960950bde2bd/users/765b58262f9743459f9d9943a30ef22a/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:58.149052 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ff17c-a16b-4236-a2ce-094b66b175aa None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:58.149329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ff17c-a16b-4236-a2ce-094b66b175aa None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:58.149601 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ff17c-a16b-4236-a2ce-094b66b175aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:58.180617 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-d50ff17c-a16b-4236-a2ce-094b66b175aa None None] MFA Rules not processed for user `765b58262f9743459f9d9943a30ef22a`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:58.181912 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-d50ff17c-a16b-4236-a2ce-094b66b175aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:58.256098 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1249/2506] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:58 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:58.265851 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e106cc0-75ea-4025-85f3-609b68d81833 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:58.339908 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e106cc0-75ea-4025-85f3-609b68d81833 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:58.345910 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de186661-beed-48ef-9450-3a41f1f431a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:58.346737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de186661-beed-48ef-9450-3a41f1f431a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:58.347116 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de186661-beed-48ef-9450-3a41f1f431a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:58.347486 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de186661-beed-48ef-9450-3a41f1f431a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:58.348237 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de186661-beed-48ef-9450-3a41f1f431a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:58.351485 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de186661-beed-48ef-9450-3a41f1f431a2 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:58.388242 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1258/2507] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:33:58 2022] POST /identity/v3/projects => generated 403 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:58.389398 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ff17c-a16b-4236-a2ce-094b66b175aa None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:58.402950 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ff17c-a16b-4236-a2ce-094b66b175aa None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:58.462779 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:58.499119 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc66711-38ea-44ca-ad19-d8b7362448cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:58.499743 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc66711-38ea-44ca-ad19-d8b7362448cb admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:58.500070 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc66711-38ea-44ca-ad19-d8b7362448cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:58.500326 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc66711-38ea-44ca-ad19-d8b7362448cb admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:58.500956 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc66711-38ea-44ca-ad19-d8b7362448cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:58.502888 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc66711-38ea-44ca-ad19-d8b7362448cb admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:58.525233 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1250/2508] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:33:58 2022] POST /identity/v3/users => generated 308 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:33:58.529509 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de186661-beed-48ef-9450-3a41f1f431a2 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:58.546994 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de186661-beed-48ef-9450-3a41f1f431a2 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:58.551701 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ef566-e4a0-4dfb-8f84-be52cc9c3bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:58.552349 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ef566-e4a0-4dfb-8f84-be52cc9c3bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:58.552629 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ef566-e4a0-4dfb-8f84-be52cc9c3bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:58.552897 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ef566-e4a0-4dfb-8f84-be52cc9c3bc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:58.553545 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162ef566-e4a0-4dfb-8f84-be52cc9c3bc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:58.555126 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162ef566-e4a0-4dfb-8f84-be52cc9c3bc9 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:58.567297 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1259/2509] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:58.571543 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc66711-38ea-44ca-ad19-d8b7362448cb admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:58.581929 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc66711-38ea-44ca-ad19-d8b7362448cb admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:58.587301 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252edf8-5813-41ef-98f1-dd48fa4d94b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:58.587987 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252edf8-5813-41ef-98f1-dd48fa4d94b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:58.588280 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252edf8-5813-41ef-98f1-dd48fa4d94b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:58.588553 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252edf8-5813-41ef-98f1-dd48fa4d94b7 admin admin] PATH_INFO: `/v3/projects/8353b178fb69412e8433036827d59979/users/361dc1a436bc4f7099dc65933101217e/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:58.605974 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f252edf8-5813-41ef-98f1-dd48fa4d94b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8353b178fb69412e8433036827d59979, user_id=361dc1a436bc4f7099dc65933101217e, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:58.608735 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f252edf8-5813-41ef-98f1-dd48fa4d94b7 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:58.623466 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1251/2510] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:33:58 2022] PUT /identity/v3/projects/8353b178fb69412e8433036827d59979/users/361dc1a436bc4f7099dc65933101217e/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:58.627521 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ef566-e4a0-4dfb-8f84-be52cc9c3bc9 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:58.639580 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ef566-e4a0-4dfb-8f84-be52cc9c3bc9 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:58.655068 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:58.720456 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314eee25-51bd-49b4-9d82-c82855ebea4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:58.721359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314eee25-51bd-49b4-9d82-c82855ebea4c admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:58.721640 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314eee25-51bd-49b4-9d82-c82855ebea4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:58.721912 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314eee25-51bd-49b4-9d82-c82855ebea4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:58.722565 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314eee25-51bd-49b4-9d82-c82855ebea4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:58.724177 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314eee25-51bd-49b4-9d82-c82855ebea4c admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:58.738478 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1260/2511] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:33:58.739595 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252edf8-5813-41ef-98f1-dd48fa4d94b7 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:58.757229 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252edf8-5813-41ef-98f1-dd48fa4d94b7 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:58.762461 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5610f4-29cb-4b1a-a76e-b5a38483c17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:58.763228 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5610f4-29cb-4b1a-a76e-b5a38483c17b admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:58.763582 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5610f4-29cb-4b1a-a76e-b5a38483c17b admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:58.763935 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5610f4-29cb-4b1a-a76e-b5a38483c17b admin admin] PATH_INFO: `/v3/projects/8353b178fb69412e8433036827d59979/users/361dc1a436bc4f7099dc65933101217e/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:58.766051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5610f4-29cb-4b1a-a76e-b5a38483c17b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8353b178fb69412e8433036827d59979, user_id=361dc1a436bc4f7099dc65933101217e, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:58.768821 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5610f4-29cb-4b1a-a76e-b5a38483c17b admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:58.784844 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1252/2512] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:33:58 2022] PUT /identity/v3/projects/8353b178fb69412e8433036827d59979/users/361dc1a436bc4f7099dc65933101217e/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:58.791230 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e2a54-6e07-4c10-a5de-347ee0487c7c None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:58.791623 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e2a54-6e07-4c10-a5de-347ee0487c7c None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:58.792039 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e2a54-6e07-4c10-a5de-347ee0487c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:58.825845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-bc1e2a54-6e07-4c10-a5de-347ee0487c7c None None] MFA Rules not processed for user `361dc1a436bc4f7099dc65933101217e`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:58.827073 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-bc1e2a54-6e07-4c10-a5de-347ee0487c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:58.903503 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1261/2513] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:58 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:58.915214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5610f4-29cb-4b1a-a76e-b5a38483c17b admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:58.917674 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-fe5610f4-29cb-4b1a-a76e-b5a38483c17b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:58.983141 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5610f4-29cb-4b1a-a76e-b5a38483c17b admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:58.987739 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933bc56f-10d2-42d7-8b66-1d83d4a4739a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:58.988712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933bc56f-10d2-42d7-8b66-1d83d4a4739a service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:58.989057 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933bc56f-10d2-42d7-8b66-1d83d4a4739a service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:58.989481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933bc56f-10d2-42d7-8b66-1d83d4a4739a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:59.000665 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933bc56f-10d2-42d7-8b66-1d83d4a4739a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:59.002112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933bc56f-10d2-42d7-8b66-1d83d4a4739a service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:59.012160 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1253/2514] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:58 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:33:59.153922 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e2a54-6e07-4c10-a5de-347ee0487c7c None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:59.224371 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e2a54-6e07-4c10-a5de-347ee0487c7c None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:59.229669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7724d0da-b7fa-435d-a278-81c81882cfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:59.230458 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724d0da-b7fa-435d-a278-81c81882cfe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:59.230744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724d0da-b7fa-435d-a278-81c81882cfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:59.231006 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724d0da-b7fa-435d-a278-81c81882cfe5 admin admin] PATH_INFO: `/v3/users/765b58262f9743459f9d9943a30ef22a` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:59.232311 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7724d0da-b7fa-435d-a278-81c81882cfe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=765b58262f9743459f9d9943a30ef22a)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:59.234245 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7724d0da-b7fa-435d-a278-81c81882cfe5 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:59.281111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:59.287654 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-7724d0da-b7fa-435d-a278-81c81882cfe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '765b58262f9743459f9d9943a30ef22a'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:59.288318 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-7724d0da-b7fa-435d-a278-81c81882cfe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '765b58262f9743459f9d9943a30ef22a'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:59.296829 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-7724d0da-b7fa-435d-a278-81c81882cfe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '765b58262f9743459f9d9943a30ef22a'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:59.313373 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-7724d0da-b7fa-435d-a278-81c81882cfe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '765b58262f9743459f9d9943a30ef22a'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:59.324243 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-7724d0da-b7fa-435d-a278-81c81882cfe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '765b58262f9743459f9d9943a30ef22a'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:59.333595 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1262/2515] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:33:59 2022] DELETE /identity/v3/users/765b58262f9743459f9d9943a30ef22a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:59.339703 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7cf8e-1d92-4390-b6f2-ad4ce90d185f None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:59.340274 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7cf8e-1d92-4390-b6f2-ad4ce90d185f None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:59.340274 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7cf8e-1d92-4390-b6f2-ad4ce90d185f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:59.360493 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-c3a7cf8e-1d92-4390-b6f2-ad4ce90d185f None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:33:59.361791 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-c3a7cf8e-1d92-4390-b6f2-ad4ce90d185f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:59.424013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1254/2516] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:33:59.433895 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7724d0da-b7fa-435d-a278-81c81882cfe5 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:59.435504 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-7724d0da-b7fa-435d-a278-81c81882cfe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:59.497112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7724d0da-b7fa-435d-a278-81c81882cfe5 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:59.500913 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93257e25-7f86-4791-adc6-49b1065e5e14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:59.501579 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93257e25-7f86-4791-adc6-49b1065e5e14 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:59.501866 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93257e25-7f86-4791-adc6-49b1065e5e14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:59.502124 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93257e25-7f86-4791-adc6-49b1065e5e14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:59.511430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93257e25-7f86-4791-adc6-49b1065e5e14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:59.512864 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93257e25-7f86-4791-adc6-49b1065e5e14 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:59.522303 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1263/2517] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:33:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:33:59.820988 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7cf8e-1d92-4390-b6f2-ad4ce90d185f None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:59.822630 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-c3a7cf8e-1d92-4390-b6f2-ad4ce90d185f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:33:59.838582 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7cf8e-1d92-4390-b6f2-ad4ce90d185f None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:33:59.845757 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f5f7c-2bc6-41a7-b6df-420f0d24460c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:33:59.846402 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f5f7c-2bc6-41a7-b6df-420f0d24460c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:33:59.846770 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f5f7c-2bc6-41a7-b6df-420f0d24460c admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:33:59.847178 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f5f7c-2bc6-41a7-b6df-420f0d24460c admin admin] PATH_INFO: `/v3/projects/35b2abd6b6c846b7bada960950bde2bd` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:33:59.848505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f5f7c-2bc6-41a7-b6df-420f0d24460c admin admin] RBAC: Authorizing `identity:delete_project(project_id=35b2abd6b6c846b7bada960950bde2bd)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:33:59.850653 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f5f7c-2bc6-41a7-b6df-420f0d24460c admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:33:59.879619 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:33:59.908532 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f94f5f7c-2bc6-41a7-b6df-420f0d24460c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35b2abd6b6c846b7bada960950bde2bd', 'request_id': 'req-f94f5f7c-2bc6-41a7-b6df-420f0d24460c'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:59.915526 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f94f5f7c-2bc6-41a7-b6df-420f0d24460c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35b2abd6b6c846b7bada960950bde2bd', 'request_id': 'req-f94f5f7c-2bc6-41a7-b6df-420f0d24460c'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:59.925475 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f94f5f7c-2bc6-41a7-b6df-420f0d24460c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35b2abd6b6c846b7bada960950bde2bd', 'request_id': 'req-f94f5f7c-2bc6-41a7-b6df-420f0d24460c'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:59.937467 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f94f5f7c-2bc6-41a7-b6df-420f0d24460c admin admin] The token cache is being invalidate because project 35b2abd6b6c846b7bada960950bde2bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:33:59.937786 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f94f5f7c-2bc6-41a7-b6df-420f0d24460c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:59.938468 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-f94f5f7c-2bc6-41a7-b6df-420f0d24460c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:33:59.940009 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1255/2518] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:33:59 2022] DELETE /identity/v3/projects/35b2abd6b6c846b7bada960950bde2bd => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:33:59.944088 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93257e25-7f86-4791-adc6-49b1065e5e14 service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:33:59.953848 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93257e25-7f86-4791-adc6-49b1065e5e14 service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:00.021302 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a3f31-139d-4322-ad71-c419ad85b5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:00.021944 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a3f31-139d-4322-ad71-c419ad85b5b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:00.022220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a3f31-139d-4322-ad71-c419ad85b5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:00.022488 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a3f31-139d-4322-ad71-c419ad85b5b4 admin admin] PATH_INFO: `/v3/users/361dc1a436bc4f7099dc65933101217e` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:00.023613 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a3f31-139d-4322-ad71-c419ad85b5b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=361dc1a436bc4f7099dc65933101217e)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:00.025603 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a3f31-139d-4322-ad71-c419ad85b5b4 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:00.083558 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8d0a3f31-139d-4322-ad71-c419ad85b5b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '361dc1a436bc4f7099dc65933101217e'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:00.084215 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8d0a3f31-139d-4322-ad71-c419ad85b5b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '361dc1a436bc4f7099dc65933101217e'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:00.092775 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8d0a3f31-139d-4322-ad71-c419ad85b5b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '361dc1a436bc4f7099dc65933101217e'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:00.099288 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:00.108404 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8d0a3f31-139d-4322-ad71-c419ad85b5b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '361dc1a436bc4f7099dc65933101217e'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:00.116749 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8d0a3f31-139d-4322-ad71-c419ad85b5b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '361dc1a436bc4f7099dc65933101217e'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:00.123348 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1264/2519] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:33:59 2022] DELETE /identity/v3/users/361dc1a436bc4f7099dc65933101217e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:00.402799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f5f7c-2bc6-41a7-b6df-420f0d24460c admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:00.405309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-f94f5f7c-2bc6-41a7-b6df-420f0d24460c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:00.423227 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f5f7c-2bc6-41a7-b6df-420f0d24460c admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:00.491306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44a1ee-4609-4ce3-ba80-e266119ab149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:00.492035 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44a1ee-4609-4ce3-ba80-e266119ab149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:00.492317 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44a1ee-4609-4ce3-ba80-e266119ab149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:00.492598 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44a1ee-4609-4ce3-ba80-e266119ab149 admin admin] PATH_INFO: `/v3/projects/8353b178fb69412e8433036827d59979` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:00.493834 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44a1ee-4609-4ce3-ba80-e266119ab149 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8353b178fb69412e8433036827d59979)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:00.496156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44a1ee-4609-4ce3-ba80-e266119ab149 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:00.544868 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-cd44a1ee-4609-4ce3-ba80-e266119ab149 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8353b178fb69412e8433036827d59979', 'request_id': 'req-cd44a1ee-4609-4ce3-ba80-e266119ab149'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:00.552818 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-cd44a1ee-4609-4ce3-ba80-e266119ab149 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8353b178fb69412e8433036827d59979', 'request_id': 'req-cd44a1ee-4609-4ce3-ba80-e266119ab149'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:00.561052 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-cd44a1ee-4609-4ce3-ba80-e266119ab149 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8353b178fb69412e8433036827d59979', 'request_id': 'req-cd44a1ee-4609-4ce3-ba80-e266119ab149'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:00.572592 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-cd44a1ee-4609-4ce3-ba80-e266119ab149 admin admin] The token cache is being invalidate because project 8353b178fb69412e8433036827d59979 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:34:00.573047 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-cd44a1ee-4609-4ce3-ba80-e266119ab149 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:00.573659 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-cd44a1ee-4609-4ce3-ba80-e266119ab149 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:00.575457 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1256/2520] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:34:00 2022] DELETE /identity/v3/projects/8353b178fb69412e8433036827d59979 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:00.580137 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b75563-a076-4f04-b436-03a52e283f5d None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:00.580557 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b75563-a076-4f04-b436-03a52e283f5d None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:00.580695 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b75563-a076-4f04-b436-03a52e283f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:00.605025 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-25b75563-a076-4f04-b436-03a52e283f5d None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:34:00.606257 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-25b75563-a076-4f04-b436-03a52e283f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:00.667217 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1265/2521] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:34:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:34:00.675724 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cd2d6-c02d-4308-a03e-9061a29171b4 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:00.676136 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cd2d6-c02d-4308-a03e-9061a29171b4 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:00.676462 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cd2d6-c02d-4308-a03e-9061a29171b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:00.696862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-f68cd2d6-c02d-4308-a03e-9061a29171b4 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:34:00.698231 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-f68cd2d6-c02d-4308-a03e-9061a29171b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:00.707061 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1257/2522] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:34:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:34:00.711714 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b75563-a076-4f04-b436-03a52e283f5d None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:00.721676 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b75563-a076-4f04-b436-03a52e283f5d None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:00.726980 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c9560f-d9fe-4e82-ab6d-5445456e8ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:00.727580 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9560f-d9fe-4e82-ab6d-5445456e8ce4 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:00.727894 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9560f-d9fe-4e82-ab6d-5445456e8ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:00.728192 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9560f-d9fe-4e82-ab6d-5445456e8ce4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:00.728837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c9560f-d9fe-4e82-ab6d-5445456e8ce4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:34:00.729131 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c9560f-d9fe-4e82-ab6d-5445456e8ce4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:00.730752 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c9560f-d9fe-4e82-ab6d-5445456e8ce4 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:00.743901 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1266/2523] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:34:00.748393 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68cd2d6-c02d-4308-a03e-9061a29171b4 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:00.762335 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68cd2d6-c02d-4308-a03e-9061a29171b4 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:00.765764 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2667a1-9b5b-484f-9d33-afd13e8c5bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:00.766581 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2667a1-9b5b-484f-9d33-afd13e8c5bcd admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:00.766843 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2667a1-9b5b-484f-9d33-afd13e8c5bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:00.767225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2667a1-9b5b-484f-9d33-afd13e8c5bcd admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:00.767959 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2667a1-9b5b-484f-9d33-afd13e8c5bcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:00.770105 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2667a1-9b5b-484f-9d33-afd13e8c5bcd admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:00.801665 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1258/2524] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:34:00 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:34:00.806307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c9560f-d9fe-4e82-ab6d-5445456e8ce4 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:00.817187 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c9560f-d9fe-4e82-ab6d-5445456e8ce4 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:00.900279 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e02f05c-d546-408f-80ea-9e0014bc0861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:00.901157 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02f05c-d546-408f-80ea-9e0014bc0861 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:00.903531 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02f05c-d546-408f-80ea-9e0014bc0861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:00.904090 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02f05c-d546-408f-80ea-9e0014bc0861 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:00.908339 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e02f05c-d546-408f-80ea-9e0014bc0861 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:00.910298 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e02f05c-d546-408f-80ea-9e0014bc0861 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:00.977725 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1267/2525] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:34:00 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:34:00.981517 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2667a1-9b5b-484f-9d33-afd13e8c5bcd admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:01.003957 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2667a1-9b5b-484f-9d33-afd13e8c5bcd admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:01.010433 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482fd80-4c16-42fd-86d0-c739d137526a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:01.011993 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482fd80-4c16-42fd-86d0-c739d137526a admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:01.012285 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482fd80-4c16-42fd-86d0-c739d137526a admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:01.012570 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482fd80-4c16-42fd-86d0-c739d137526a admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:01.013297 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482fd80-4c16-42fd-86d0-c739d137526a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:01.015169 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482fd80-4c16-42fd-86d0-c739d137526a admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:01.031201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1259/2526] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:34:01.035178 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e02f05c-d546-408f-80ea-9e0014bc0861 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:01.051364 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e02f05c-d546-408f-80ea-9e0014bc0861 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:01.083409 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c31f68-c0e8-4c71-a1a1-11660fccdb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:01.084454 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c31f68-c0e8-4c71-a1a1-11660fccdb71 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:01.084901 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c31f68-c0e8-4c71-a1a1-11660fccdb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:01.085416 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c31f68-c0e8-4c71-a1a1-11660fccdb71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:01.087217 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c31f68-c0e8-4c71-a1a1-11660fccdb71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:01.089734 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c31f68-c0e8-4c71-a1a1-11660fccdb71 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:01.137879 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1268/2527] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:34:01.142341 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482fd80-4c16-42fd-86d0-c739d137526a admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:01.147952 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:01.161292 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482fd80-4c16-42fd-86d0-c739d137526a admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:01.166126 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7764ed39-de78-4852-8cc8-e3720fe75912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:01.167058 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764ed39-de78-4852-8cc8-e3720fe75912 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:01.167796 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764ed39-de78-4852-8cc8-e3720fe75912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:01.168110 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764ed39-de78-4852-8cc8-e3720fe75912 admin admin] PATH_INFO: `/v3/projects/98c1de443dd14892a089e9f9196178a4/users/c64118e6e66e445e8fd0b17dfc80c702/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:01.185795 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7764ed39-de78-4852-8cc8-e3720fe75912 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98c1de443dd14892a089e9f9196178a4, user_id=c64118e6e66e445e8fd0b17dfc80c702, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:01.188682 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7764ed39-de78-4852-8cc8-e3720fe75912 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:01.207961 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1260/2528] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:34:01 2022] PUT /identity/v3/projects/98c1de443dd14892a089e9f9196178a4/users/c64118e6e66e445e8fd0b17dfc80c702/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:01.217690 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce77855-fffd-48ed-8cdb-31e28c2c82c5 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:01.217980 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce77855-fffd-48ed-8cdb-31e28c2c82c5 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:01.218252 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce77855-fffd-48ed-8cdb-31e28c2c82c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:01.256612 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-bce77855-fffd-48ed-8cdb-31e28c2c82c5 None None] MFA Rules not processed for user `c64118e6e66e445e8fd0b17dfc80c702`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:34:01.257957 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-bce77855-fffd-48ed-8cdb-31e28c2c82c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:01.321557 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:01.358827 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1269/2529] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:34:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:34:01.371176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7764ed39-de78-4852-8cc8-e3720fe75912 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:01.373050 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-7764ed39-de78-4852-8cc8-e3720fe75912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:01.430561 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7764ed39-de78-4852-8cc8-e3720fe75912 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:01.434711 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8b0e1-ba12-452f-b87f-05b3bef14818 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f1bf93cff6c46e0801bb827203cb5ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:01.435328 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8b0e1-ba12-452f-b87f-05b3bef14818 service glance] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:01.435588 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8b0e1-ba12-452f-b87f-05b3bef14818 service glance] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:01.435953 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8b0e1-ba12-452f-b87f-05b3bef14818 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:01.444959 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd8b0e1-ba12-452f-b87f-05b3bef14818 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:01.446408 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd8b0e1-ba12-452f-b87f-05b3bef14818 service glance] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:01.455940 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1261/2530] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:34:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:34:05.206794 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce77855-fffd-48ed-8cdb-31e28c2c82c5 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:05.208335 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-bce77855-fffd-48ed-8cdb-31e28c2c82c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:05.281971 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce77855-fffd-48ed-8cdb-31e28c2c82c5 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:05.286989 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd070f1-3219-4cde-befb-fe08870f8e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:05.287622 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd070f1-3219-4cde-befb-fe08870f8e03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:05.287897 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd070f1-3219-4cde-befb-fe08870f8e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:05.288190 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd070f1-3219-4cde-befb-fe08870f8e03 admin admin] PATH_INFO: `/v3/users/59083982a8794b69a93e6944149ac65a` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:05.289345 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd070f1-3219-4cde-befb-fe08870f8e03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59083982a8794b69a93e6944149ac65a)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:05.291537 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd070f1-3219-4cde-befb-fe08870f8e03 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:05.357464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-afd070f1-3219-4cde-befb-fe08870f8e03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59083982a8794b69a93e6944149ac65a'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:05.358203 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-afd070f1-3219-4cde-befb-fe08870f8e03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59083982a8794b69a93e6944149ac65a'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:05.366932 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-afd070f1-3219-4cde-befb-fe08870f8e03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59083982a8794b69a93e6944149ac65a'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:05.379439 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-afd070f1-3219-4cde-befb-fe08870f8e03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59083982a8794b69a93e6944149ac65a'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:05.387399 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-afd070f1-3219-4cde-befb-fe08870f8e03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59083982a8794b69a93e6944149ac65a'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:05.395480 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1270/2531] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:34:05 2022] DELETE /identity/v3/users/59083982a8794b69a93e6944149ac65a => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:05.545121 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8b0e1-ba12-452f-b87f-05b3bef14818 service glance] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:05.547233 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-7cd8b0e1-ba12-452f-b87f-05b3bef14818 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:05.593578 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd070f1-3219-4cde-befb-fe08870f8e03 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:05.595183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-afd070f1-3219-4cde-befb-fe08870f8e03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:05.618186 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8b0e1-ba12-452f-b87f-05b3bef14818 service glance] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:05.661493 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:05.677208 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd070f1-3219-4cde-befb-fe08870f8e03 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:05.681232 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536bfc2f-7cad-4614-86cc-409a0c57190e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:05.682023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536bfc2f-7cad-4614-86cc-409a0c57190e service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:05.682142 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8eaee4-15da-466f-bb44-85426ca36fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:05.682935 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8eaee4-15da-466f-bb44-85426ca36fcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:05.683177 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536bfc2f-7cad-4614-86cc-409a0c57190e service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:05.683277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8eaee4-15da-466f-bb44-85426ca36fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:05.683461 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536bfc2f-7cad-4614-86cc-409a0c57190e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:05.683555 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8eaee4-15da-466f-bb44-85426ca36fcf admin admin] PATH_INFO: `/v3/projects/16e0c2b8e2a8400b90fc58de9b69a5b1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:05.684623 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8eaee4-15da-466f-bb44-85426ca36fcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=16e0c2b8e2a8400b90fc58de9b69a5b1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:05.685257 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-536bfc2f-7cad-4614-86cc-409a0c57190e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:05.686524 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8eaee4-15da-466f-bb44-85426ca36fcf admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:05.698176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536bfc2f-7cad-4614-86cc-409a0c57190e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:05.699723 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536bfc2f-7cad-4614-86cc-409a0c57190e service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:05.710193 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1262/2532] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:34:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:34:05.737924 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-5e8eaee4-15da-466f-bb44-85426ca36fcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16e0c2b8e2a8400b90fc58de9b69a5b1', 'request_id': 'req-5e8eaee4-15da-466f-bb44-85426ca36fcf'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:05.746662 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-5e8eaee4-15da-466f-bb44-85426ca36fcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16e0c2b8e2a8400b90fc58de9b69a5b1', 'request_id': 'req-5e8eaee4-15da-466f-bb44-85426ca36fcf'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:05.758909 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-5e8eaee4-15da-466f-bb44-85426ca36fcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16e0c2b8e2a8400b90fc58de9b69a5b1', 'request_id': 'req-5e8eaee4-15da-466f-bb44-85426ca36fcf'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:05.772656 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-5e8eaee4-15da-466f-bb44-85426ca36fcf admin admin] The token cache is being invalidate because project 16e0c2b8e2a8400b90fc58de9b69a5b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:34:05.772993 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-5e8eaee4-15da-466f-bb44-85426ca36fcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:05.773648 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-5e8eaee4-15da-466f-bb44-85426ca36fcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:05.775256 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1271/2533] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:34:05 2022] DELETE /identity/v3/projects/16e0c2b8e2a8400b90fc58de9b69a5b1 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:34:05.779789 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536bfc2f-7cad-4614-86cc-409a0c57190e service neutron] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:05.790072 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536bfc2f-7cad-4614-86cc-409a0c57190e service neutron] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:05.836406 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:05.860297 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2c987-737f-470e-aeac-5f95e785ea41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:05.860923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2c987-737f-470e-aeac-5f95e785ea41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:05.861199 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2c987-737f-470e-aeac-5f95e785ea41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:05.861465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2c987-737f-470e-aeac-5f95e785ea41 admin admin] PATH_INFO: `/v3/users/7a9994c849634a848073ac56e60b96f5` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:05.862589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2c987-737f-470e-aeac-5f95e785ea41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a9994c849634a848073ac56e60b96f5)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:05.864604 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2c987-737f-470e-aeac-5f95e785ea41 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:05.917885 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-4ac2c987-737f-470e-aeac-5f95e785ea41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a9994c849634a848073ac56e60b96f5'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:05.931380 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-4ac2c987-737f-470e-aeac-5f95e785ea41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a9994c849634a848073ac56e60b96f5'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:05.945281 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-4ac2c987-737f-470e-aeac-5f95e785ea41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a9994c849634a848073ac56e60b96f5'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:05.946049 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-4ac2c987-737f-470e-aeac-5f95e785ea41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a9994c849634a848073ac56e60b96f5'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:05.953407 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-4ac2c987-737f-470e-aeac-5f95e785ea41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a9994c849634a848073ac56e60b96f5'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:05.961180 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1263/2534] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:34:05 2022] DELETE /identity/v3/users/7a9994c849634a848073ac56e60b96f5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:06.197042 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8eaee4-15da-466f-bb44-85426ca36fcf admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:06.198597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-5e8eaee4-15da-466f-bb44-85426ca36fcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:06.214143 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8eaee4-15da-466f-bb44-85426ca36fcf admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:06.282864 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd5488c-7917-4acf-9158-03d599ca0b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:06.283515 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5488c-7917-4acf-9158-03d599ca0b00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:06.283791 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5488c-7917-4acf-9158-03d599ca0b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:06.284086 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5488c-7917-4acf-9158-03d599ca0b00 admin admin] PATH_INFO: `/v3/projects/fd86d315464043dd8767ff1bfd7cd3d1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:06.285248 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd5488c-7917-4acf-9158-03d599ca0b00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd86d315464043dd8767ff1bfd7cd3d1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:06.287211 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd5488c-7917-4acf-9158-03d599ca0b00 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:06.342448 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ebd5488c-7917-4acf-9158-03d599ca0b00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd86d315464043dd8767ff1bfd7cd3d1', 'request_id': 'req-ebd5488c-7917-4acf-9158-03d599ca0b00'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:06.351320 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ebd5488c-7917-4acf-9158-03d599ca0b00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd86d315464043dd8767ff1bfd7cd3d1', 'request_id': 'req-ebd5488c-7917-4acf-9158-03d599ca0b00'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:06.359902 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ebd5488c-7917-4acf-9158-03d599ca0b00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd86d315464043dd8767ff1bfd7cd3d1', 'request_id': 'req-ebd5488c-7917-4acf-9158-03d599ca0b00'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:06.371184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ebd5488c-7917-4acf-9158-03d599ca0b00 admin admin] The token cache is being invalidate because project fd86d315464043dd8767ff1bfd7cd3d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:34:06.371472 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ebd5488c-7917-4acf-9158-03d599ca0b00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:06.372168 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ebd5488c-7917-4acf-9158-03d599ca0b00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:06.373681 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1272/2535] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:34:06 2022] DELETE /identity/v3/projects/fd86d315464043dd8767ff1bfd7cd3d1 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:06.378753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bd4d0-c6ef-47d3-b6bd-dc6f14402b6d None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:06.379065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bd4d0-c6ef-47d3-b6bd-dc6f14402b6d None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:06.379337 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bd4d0-c6ef-47d3-b6bd-dc6f14402b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:06.400243 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-286bd4d0-c6ef-47d3-b6bd-dc6f14402b6d None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:34:06.401635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-286bd4d0-c6ef-47d3-b6bd-dc6f14402b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:06.474988 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1264/2536] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:34:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:34:06.478856 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072b01f-7e3d-4bd8-932c-3a313990ddba None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:06.479169 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072b01f-7e3d-4bd8-932c-3a313990ddba None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:06.480073 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072b01f-7e3d-4bd8-932c-3a313990ddba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:06.501900 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-4072b01f-7e3d-4bd8-932c-3a313990ddba None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:34:06.503169 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-4072b01f-7e3d-4bd8-932c-3a313990ddba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:06.512947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1273/2537] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:34:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:34:06.517447 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286bd4d0-c6ef-47d3-b6bd-dc6f14402b6d None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:06.527085 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286bd4d0-c6ef-47d3-b6bd-dc6f14402b6d None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:06.531757 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2617afc6-66e2-4714-bac5-7d7c9b76fcd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:06.532391 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617afc6-66e2-4714-bac5-7d7c9b76fcd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:06.532661 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617afc6-66e2-4714-bac5-7d7c9b76fcd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:06.532925 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617afc6-66e2-4714-bac5-7d7c9b76fcd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:06.533565 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2617afc6-66e2-4714-bac5-7d7c9b76fcd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:34:06.533855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2617afc6-66e2-4714-bac5-7d7c9b76fcd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:06.535474 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2617afc6-66e2-4714-bac5-7d7c9b76fcd9 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:06.551017 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1265/2538] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:34:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:34:06.552368 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4072b01f-7e3d-4bd8-932c-3a313990ddba None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:06.563320 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4072b01f-7e3d-4bd8-932c-3a313990ddba None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:06.568132 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40709b5-b6df-4eff-a04e-1afd9ba36728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:06.568742 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40709b5-b6df-4eff-a04e-1afd9ba36728 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:06.569016 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40709b5-b6df-4eff-a04e-1afd9ba36728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:06.569311 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40709b5-b6df-4eff-a04e-1afd9ba36728 admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:06.569916 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40709b5-b6df-4eff-a04e-1afd9ba36728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:06.571804 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40709b5-b6df-4eff-a04e-1afd9ba36728 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:06.576796 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:06.605337 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1274/2539] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:34:06 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:34:06.609781 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2617afc6-66e2-4714-bac5-7d7c9b76fcd9 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:06.620836 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2617afc6-66e2-4714-bac5-7d7c9b76fcd9 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:06.669388 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:06.695325 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96721324-b1d6-407e-bb6d-70dd7e94a36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:06.696139 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96721324-b1d6-407e-bb6d-70dd7e94a36b admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:06.696440 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96721324-b1d6-407e-bb6d-70dd7e94a36b admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:06.696573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96721324-b1d6-407e-bb6d-70dd7e94a36b admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:06.697217 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96721324-b1d6-407e-bb6d-70dd7e94a36b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:06.699305 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96721324-b1d6-407e-bb6d-70dd7e94a36b admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:06.722904 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1266/2540] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:34:06 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:34:06.727696 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40709b5-b6df-4eff-a04e-1afd9ba36728 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:06.748006 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40709b5-b6df-4eff-a04e-1afd9ba36728 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:06.754993 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c165b-c0ee-448f-ae3d-02e429bdabdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:06.755627 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c165b-c0ee-448f-ae3d-02e429bdabdc admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:06.755883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c165b-c0ee-448f-ae3d-02e429bdabdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:06.756222 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c165b-c0ee-448f-ae3d-02e429bdabdc admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:06.756906 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c165b-c0ee-448f-ae3d-02e429bdabdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:06.758571 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c165b-c0ee-448f-ae3d-02e429bdabdc admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:06.769902 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1275/2541] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:34:06.773799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96721324-b1d6-407e-bb6d-70dd7e94a36b admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:06.789908 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96721324-b1d6-407e-bb6d-70dd7e94a36b admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:06.795239 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01bee7-905b-4999-8f90-c6f3dbbd928f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:06.796225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01bee7-905b-4999-8f90-c6f3dbbd928f admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:06.796609 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01bee7-905b-4999-8f90-c6f3dbbd928f admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:06.796982 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01bee7-905b-4999-8f90-c6f3dbbd928f admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:06.797743 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01bee7-905b-4999-8f90-c6f3dbbd928f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:06.799513 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01bee7-905b-4999-8f90-c6f3dbbd928f admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:06.815995 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1267/2542] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:34:06.816345 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c165b-c0ee-448f-ae3d-02e429bdabdc admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:06.827042 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c165b-c0ee-448f-ae3d-02e429bdabdc admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:06.832286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b890879-b53d-4f06-b89c-44ff53885edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:06.832965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b890879-b53d-4f06-b89c-44ff53885edb admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:06.833262 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b890879-b53d-4f06-b89c-44ff53885edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:06.833554 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b890879-b53d-4f06-b89c-44ff53885edb admin admin] PATH_INFO: `/v3/projects/c285b3c892ec494dac9fc255c4178606/users/b7aa17006bd64559b89f651f532312d3/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:06.850842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b890879-b53d-4f06-b89c-44ff53885edb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c285b3c892ec494dac9fc255c4178606, user_id=b7aa17006bd64559b89f651f532312d3, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:06.853649 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b890879-b53d-4f06-b89c-44ff53885edb admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:06.873166 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1276/2543] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:34:06 2022] PUT /identity/v3/projects/c285b3c892ec494dac9fc255c4178606/users/b7aa17006bd64559b89f651f532312d3/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:06.884808 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218ba42-8afc-4a2f-bdd0-b57d5f314ef6 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:06.885105 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218ba42-8afc-4a2f-bdd0-b57d5f314ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:06.885363 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218ba42-8afc-4a2f-bdd0-b57d5f314ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:06.915629 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-e218ba42-8afc-4a2f-bdd0-b57d5f314ef6 None None] MFA Rules not processed for user `b7aa17006bd64559b89f651f532312d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:34:06.916995 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-e218ba42-8afc-4a2f-bdd0-b57d5f314ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:06.989536 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1268/2544] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:34:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:34:06.994923 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b890879-b53d-4f06-b89c-44ff53885edb admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:07.062285 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b890879-b53d-4f06-b89c-44ff53885edb admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:07.067944 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74305f4e-1a7a-461e-b661-69db28475395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:07.068603 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74305f4e-1a7a-461e-b661-69db28475395 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:07.068910 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74305f4e-1a7a-461e-b661-69db28475395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:07.069178 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74305f4e-1a7a-461e-b661-69db28475395 admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:07.069856 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74305f4e-1a7a-461e-b661-69db28475395 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:07.071894 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74305f4e-1a7a-461e-b661-69db28475395 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:07.101600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1277/2545] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:34:06 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:34:07.106148 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218ba42-8afc-4a2f-bdd0-b57d5f314ef6 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:07.117770 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218ba42-8afc-4a2f-bdd0-b57d5f314ef6 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:07.186442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a326d6d-7e71-4a22-a927-46f81bcf39fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:07.187180 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a326d6d-7e71-4a22-a927-46f81bcf39fd admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:07.187465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a326d6d-7e71-4a22-a927-46f81bcf39fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:07.187732 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a326d6d-7e71-4a22-a927-46f81bcf39fd admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:07.188451 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a326d6d-7e71-4a22-a927-46f81bcf39fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:07.195138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a326d6d-7e71-4a22-a927-46f81bcf39fd admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:07.227516 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1269/2546] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:34:07 2022] POST /identity/v3/users => generated 299 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:34:07.230382 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74305f4e-1a7a-461e-b661-69db28475395 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:07.247598 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74305f4e-1a7a-461e-b661-69db28475395 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:07.252409 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074cb53b-99b6-44b5-80b5-e78efca94ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:07.253022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074cb53b-99b6-44b5-80b5-e78efca94ae0 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:07.253299 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074cb53b-99b6-44b5-80b5-e78efca94ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:07.253571 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074cb53b-99b6-44b5-80b5-e78efca94ae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:07.254213 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074cb53b-99b6-44b5-80b5-e78efca94ae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:07.255809 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074cb53b-99b6-44b5-80b5-e78efca94ae0 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:07.267232 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1278/2547] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:34:07.271899 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a326d6d-7e71-4a22-a927-46f81bcf39fd admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:07.290027 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a326d6d-7e71-4a22-a927-46f81bcf39fd admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:07.297511 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa37531-7a13-42c0-be23-a35a5d23df81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:07.298331 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa37531-7a13-42c0-be23-a35a5d23df81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:07.298684 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa37531-7a13-42c0-be23-a35a5d23df81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:07.299472 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa37531-7a13-42c0-be23-a35a5d23df81 admin admin] PATH_INFO: `/v3/projects/bed0b66099034c1c84b3477e118e58b2/users/cf7f923c92014fda85af3e30cb36d38a/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:07.321601 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa37531-7a13-42c0-be23-a35a5d23df81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bed0b66099034c1c84b3477e118e58b2, user_id=cf7f923c92014fda85af3e30cb36d38a, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:07.324692 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa37531-7a13-42c0-be23-a35a5d23df81 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:07.343653 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1270/2548] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:34:07 2022] PUT /identity/v3/projects/bed0b66099034c1c84b3477e118e58b2/users/cf7f923c92014fda85af3e30cb36d38a/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:07.347581 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074cb53b-99b6-44b5-80b5-e78efca94ae0 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:07.359272 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074cb53b-99b6-44b5-80b5-e78efca94ae0 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:07.438851 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a179be-a9f0-440a-bc7c-dc5ffb4f78d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:07.439480 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a179be-a9f0-440a-bc7c-dc5ffb4f78d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:07.439765 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a179be-a9f0-440a-bc7c-dc5ffb4f78d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:07.440086 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a179be-a9f0-440a-bc7c-dc5ffb4f78d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:07.440735 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a179be-a9f0-440a-bc7c-dc5ffb4f78d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:07.442402 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a179be-a9f0-440a-bc7c-dc5ffb4f78d9 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:07.454376 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1279/2549] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:34:07.458669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa37531-7a13-42c0-be23-a35a5d23df81 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:07.470077 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa37531-7a13-42c0-be23-a35a5d23df81 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:07.475803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b905d-5aa4-49ee-a369-77fb8df5af60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:07.476621 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b905d-5aa4-49ee-a369-77fb8df5af60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:07.476966 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b905d-5aa4-49ee-a369-77fb8df5af60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:07.477247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b905d-5aa4-49ee-a369-77fb8df5af60 admin admin] PATH_INFO: `/v3/projects/bed0b66099034c1c84b3477e118e58b2/users/cf7f923c92014fda85af3e30cb36d38a/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:07.483774 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20b905d-5aa4-49ee-a369-77fb8df5af60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bed0b66099034c1c84b3477e118e58b2, user_id=cf7f923c92014fda85af3e30cb36d38a, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:07.486497 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20b905d-5aa4-49ee-a369-77fb8df5af60 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:07.498328 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1271/2550] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:34:07 2022] PUT /identity/v3/projects/bed0b66099034c1c84b3477e118e58b2/users/cf7f923c92014fda85af3e30cb36d38a/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:34:07.508989 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e4be1-c5a3-4e7c-bd6e-30eb495737f6 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:07.509349 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e4be1-c5a3-4e7c-bd6e-30eb495737f6 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:07.509562 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e4be1-c5a3-4e7c-bd6e-30eb495737f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:07.547894 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-cb2e4be1-c5a3-4e7c-bd6e-30eb495737f6 None None] MFA Rules not processed for user `cf7f923c92014fda85af3e30cb36d38a`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:34:07.549179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-cb2e4be1-c5a3-4e7c-bd6e-30eb495737f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:07.561481 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:07.641093 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1280/2551] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:34:07 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:34:07.656246 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b905d-5aa4-49ee-a369-77fb8df5af60 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:07.658101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-e20b905d-5aa4-49ee-a369-77fb8df5af60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:07.669346 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:07.730484 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b905d-5aa4-49ee-a369-77fb8df5af60 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:07.739987 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de15ea3b-a5d9-4950-8de7-23433ccaa075 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:07.740669 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15ea3b-a5d9-4950-8de7-23433ccaa075 service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:07.740942 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15ea3b-a5d9-4950-8de7-23433ccaa075 service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:07.741203 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15ea3b-a5d9-4950-8de7-23433ccaa075 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:07.816589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de15ea3b-a5d9-4950-8de7-23433ccaa075 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:07.818282 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de15ea3b-a5d9-4950-8de7-23433ccaa075 service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:07.829126 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1272/2552] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:34:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:34:09.217004 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e4be1-c5a3-4e7c-bd6e-30eb495737f6 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:09.219060 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-cb2e4be1-c5a3-4e7c-bd6e-30eb495737f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:09.292681 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e4be1-c5a3-4e7c-bd6e-30eb495737f6 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:09.297917 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b9b08-f096-4769-b0fb-a79b87cfc7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:09.298613 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b9b08-f096-4769-b0fb-a79b87cfc7da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:09.298976 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b9b08-f096-4769-b0fb-a79b87cfc7da admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:09.299142 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b9b08-f096-4769-b0fb-a79b87cfc7da admin admin] PATH_INFO: `/v3/users/c8fa0859e5c14c53a5a2473c95efe152` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:09.300572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b9b08-f096-4769-b0fb-a79b87cfc7da admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8fa0859e5c14c53a5a2473c95efe152)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:09.302498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b9b08-f096-4769-b0fb-a79b87cfc7da admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:09.372142 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-3e6b9b08-f096-4769-b0fb-a79b87cfc7da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8fa0859e5c14c53a5a2473c95efe152'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:09.372797 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-3e6b9b08-f096-4769-b0fb-a79b87cfc7da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8fa0859e5c14c53a5a2473c95efe152'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:09.381176 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-3e6b9b08-f096-4769-b0fb-a79b87cfc7da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8fa0859e5c14c53a5a2473c95efe152'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:09.393881 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-3e6b9b08-f096-4769-b0fb-a79b87cfc7da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8fa0859e5c14c53a5a2473c95efe152'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:09.401349 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-3e6b9b08-f096-4769-b0fb-a79b87cfc7da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8fa0859e5c14c53a5a2473c95efe152'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:09.407580 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1281/2553] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:34:09 2022] DELETE /identity/v3/users/c8fa0859e5c14c53a5a2473c95efe152 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:09.602382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de15ea3b-a5d9-4950-8de7-23433ccaa075 service cinder] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:09.621895 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de15ea3b-a5d9-4950-8de7-23433ccaa075 service cinder] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:09.721804 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93874e-39e0-41d0-9475-13699473bb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:09.722456 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93874e-39e0-41d0-9475-13699473bb4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:09.722740 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93874e-39e0-41d0-9475-13699473bb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:09.723014 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93874e-39e0-41d0-9475-13699473bb4a admin admin] PATH_INFO: `/v3/projects/fa6589185abe4af1b3d0cba6f0699538` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:09.724247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93874e-39e0-41d0-9475-13699473bb4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa6589185abe4af1b3d0cba6f0699538)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:09.726502 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93874e-39e0-41d0-9475-13699473bb4a admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:09.779939 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9b93874e-39e0-41d0-9475-13699473bb4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa6589185abe4af1b3d0cba6f0699538', 'request_id': 'req-9b93874e-39e0-41d0-9475-13699473bb4a'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:09.786168 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9b93874e-39e0-41d0-9475-13699473bb4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa6589185abe4af1b3d0cba6f0699538', 'request_id': 'req-9b93874e-39e0-41d0-9475-13699473bb4a'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:09.794556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9b93874e-39e0-41d0-9475-13699473bb4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa6589185abe4af1b3d0cba6f0699538', 'request_id': 'req-9b93874e-39e0-41d0-9475-13699473bb4a'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:09.796832 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:09.808529 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9b93874e-39e0-41d0-9475-13699473bb4a admin admin] The token cache is being invalidate because project fa6589185abe4af1b3d0cba6f0699538 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:34:09.808821 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9b93874e-39e0-41d0-9475-13699473bb4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:09.809495 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9b93874e-39e0-41d0-9475-13699473bb4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:09.811444 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1273/2554] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:34:09 2022] DELETE /identity/v3/projects/fa6589185abe4af1b3d0cba6f0699538 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:09.816654 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e7875-e2fe-47d3-9378-2a35abaa81a2 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:09.816946 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e7875-e2fe-47d3-9378-2a35abaa81a2 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:09.817220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e7875-e2fe-47d3-9378-2a35abaa81a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:09.837636 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-6b7e7875-e2fe-47d3-9378-2a35abaa81a2 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:34:09.838966 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-6b7e7875-e2fe-47d3-9378-2a35abaa81a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:09.859695 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:09.907036 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1282/2555] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:34:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:34:09.915846 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd95ff-5f5b-499a-99f9-7c36ddef2598 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:09.916160 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd95ff-5f5b-499a-99f9-7c36ddef2598 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:09.916427 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd95ff-5f5b-499a-99f9-7c36ddef2598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:09.936946 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-bbfd95ff-5f5b-499a-99f9-7c36ddef2598 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:34:09.938136 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-bbfd95ff-5f5b-499a-99f9-7c36ddef2598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:09.946082 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1274/2556] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:34:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:34:09.950801 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e7875-e2fe-47d3-9378-2a35abaa81a2 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:09.962027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e7875-e2fe-47d3-9378-2a35abaa81a2 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:09.967836 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668af93a-fb1a-4733-8f1d-f0d1dcffe619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:09.968493 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668af93a-fb1a-4733-8f1d-f0d1dcffe619 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:09.968772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668af93a-fb1a-4733-8f1d-f0d1dcffe619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:09.969047 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668af93a-fb1a-4733-8f1d-f0d1dcffe619 admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:09.969915 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668af93a-fb1a-4733-8f1d-f0d1dcffe619 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:34:09.970415 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668af93a-fb1a-4733-8f1d-f0d1dcffe619 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:09.972595 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668af93a-fb1a-4733-8f1d-f0d1dcffe619 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:09.990748 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1283/2557] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:34:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:34:09.995480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd95ff-5f5b-499a-99f9-7c36ddef2598 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:10.005307 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd95ff-5f5b-499a-99f9-7c36ddef2598 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:10.011389 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84785038-a254-45d1-966e-b1deecf52e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:10.012023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84785038-a254-45d1-966e-b1deecf52e35 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:10.012281 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84785038-a254-45d1-966e-b1deecf52e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:10.012543 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84785038-a254-45d1-966e-b1deecf52e35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:10.013162 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84785038-a254-45d1-966e-b1deecf52e35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:10.015054 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84785038-a254-45d1-966e-b1deecf52e35 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:10.046045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1275/2558] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:34:09 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:34:10.052694 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668af93a-fb1a-4733-8f1d-f0d1dcffe619 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:10.069763 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668af93a-fb1a-4733-8f1d-f0d1dcffe619 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:10.136821 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c72af3-b5c3-4c13-962e-37d15816a8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:10.137453 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c72af3-b5c3-4c13-962e-37d15816a8f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:10.137737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c72af3-b5c3-4c13-962e-37d15816a8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:10.137995 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c72af3-b5c3-4c13-962e-37d15816a8f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:10.138618 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c72af3-b5c3-4c13-962e-37d15816a8f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:10.140621 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c72af3-b5c3-4c13-962e-37d15816a8f7 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:10.163721 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1284/2559] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:34:10 2022] POST /identity/v3/users => generated 284 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:34:10.168194 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84785038-a254-45d1-966e-b1deecf52e35 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:10.179251 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84785038-a254-45d1-966e-b1deecf52e35 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:10.184489 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b9c1ac-1e80-44e3-9a56-27fc1fb39667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:10.185101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9c1ac-1e80-44e3-9a56-27fc1fb39667 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:10.185374 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9c1ac-1e80-44e3-9a56-27fc1fb39667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:10.185642 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9c1ac-1e80-44e3-9a56-27fc1fb39667 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:10.186377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b9c1ac-1e80-44e3-9a56-27fc1fb39667 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:10.188011 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b9c1ac-1e80-44e3-9a56-27fc1fb39667 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:10.198848 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1276/2560] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:34:10.203475 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c72af3-b5c3-4c13-962e-37d15816a8f7 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:10.219974 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c72af3-b5c3-4c13-962e-37d15816a8f7 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:10.226545 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f5956-e9e9-41d3-98c0-f2b71bcab5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:10.227166 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f5956-e9e9-41d3-98c0-f2b71bcab5ce admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:10.227446 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f5956-e9e9-41d3-98c0-f2b71bcab5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:10.227715 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f5956-e9e9-41d3-98c0-f2b71bcab5ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:10.228417 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f5956-e9e9-41d3-98c0-f2b71bcab5ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:10.230104 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f5956-e9e9-41d3-98c0-f2b71bcab5ce admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:10.242223 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1285/2561] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:34:10.246531 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b9c1ac-1e80-44e3-9a56-27fc1fb39667 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:10.257779 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b9c1ac-1e80-44e3-9a56-27fc1fb39667 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:10.267889 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c093703-db83-4548-9a90-a7d90bf85cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:10.268743 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c093703-db83-4548-9a90-a7d90bf85cf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:10.269025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c093703-db83-4548-9a90-a7d90bf85cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:10.269310 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c093703-db83-4548-9a90-a7d90bf85cf0 admin admin] PATH_INFO: `/v3/projects/cc551179c4fd40ef93d92d1a1c8e11bc/users/57c565869ac14ef0bf6e5f42273523e8/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:10.285790 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c093703-db83-4548-9a90-a7d90bf85cf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc551179c4fd40ef93d92d1a1c8e11bc, user_id=57c565869ac14ef0bf6e5f42273523e8, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:10.288533 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c093703-db83-4548-9a90-a7d90bf85cf0 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:10.302750 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1277/2562] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:34:10 2022] PUT /identity/v3/projects/cc551179c4fd40ef93d92d1a1c8e11bc/users/57c565869ac14ef0bf6e5f42273523e8/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:10.313450 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe274fa-38ec-4ffd-8a98-bc648da08eab None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:10.313735 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe274fa-38ec-4ffd-8a98-bc648da08eab None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:10.314006 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe274fa-38ec-4ffd-8a98-bc648da08eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:10.343212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-abe274fa-38ec-4ffd-8a98-bc648da08eab None None] MFA Rules not processed for user `57c565869ac14ef0bf6e5f42273523e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:34:10.344491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-abe274fa-38ec-4ffd-8a98-bc648da08eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:10.417294 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1286/2563] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:34:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:34:10.428026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c093703-db83-4548-9a90-a7d90bf85cf0 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:10.429749 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-8c093703-db83-4548-9a90-a7d90bf85cf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:10.491096 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c093703-db83-4548-9a90-a7d90bf85cf0 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:10.495543 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d109f89-cbcd-457d-aa71-fc809e432390 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:10.496246 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d109f89-cbcd-457d-aa71-fc809e432390 service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:10.496526 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d109f89-cbcd-457d-aa71-fc809e432390 service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:10.496908 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d109f89-cbcd-457d-aa71-fc809e432390 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:10.506269 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d109f89-cbcd-457d-aa71-fc809e432390 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:10.507726 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d109f89-cbcd-457d-aa71-fc809e432390 service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:10.518068 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1278/2564] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:34:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:34:12.225867 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe274fa-38ec-4ffd-8a98-bc648da08eab None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:12.228321 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-abe274fa-38ec-4ffd-8a98-bc648da08eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:12.243837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe274fa-38ec-4ffd-8a98-bc648da08eab None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:12.249774 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69425788-a953-4f3b-a6da-35f12f8a938d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:12.250428 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69425788-a953-4f3b-a6da-35f12f8a938d service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:12.251716 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69425788-a953-4f3b-a6da-35f12f8a938d service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:12.251716 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69425788-a953-4f3b-a6da-35f12f8a938d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:12.253242 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-69425788-a953-4f3b-a6da-35f12f8a938d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:12.273213 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:12.336844 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69425788-a953-4f3b-a6da-35f12f8a938d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:12.338300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69425788-a953-4f3b-a6da-35f12f8a938d service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:12.348580 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1287/2565] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:34:12 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:34:17.315743 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d109f89-cbcd-457d-aa71-fc809e432390 service cinder] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:17.317463 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-2d109f89-cbcd-457d-aa71-fc809e432390 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:17.388270 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d109f89-cbcd-457d-aa71-fc809e432390 service cinder] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:17.396565 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755bda39-dc39-41b5-ae38-e9abbda1d1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:17.397524 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755bda39-dc39-41b5-ae38-e9abbda1d1cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:17.397867 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755bda39-dc39-41b5-ae38-e9abbda1d1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:17.398278 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755bda39-dc39-41b5-ae38-e9abbda1d1cf admin admin] PATH_INFO: `/v3/users/b7aa17006bd64559b89f651f532312d3` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:17.400442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755bda39-dc39-41b5-ae38-e9abbda1d1cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7aa17006bd64559b89f651f532312d3)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:17.403667 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755bda39-dc39-41b5-ae38-e9abbda1d1cf admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:17.441237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:17.454856 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-755bda39-dc39-41b5-ae38-e9abbda1d1cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7aa17006bd64559b89f651f532312d3'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:17.464376 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-755bda39-dc39-41b5-ae38-e9abbda1d1cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7aa17006bd64559b89f651f532312d3'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:17.477748 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-755bda39-dc39-41b5-ae38-e9abbda1d1cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7aa17006bd64559b89f651f532312d3'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:17.478452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-755bda39-dc39-41b5-ae38-e9abbda1d1cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7aa17006bd64559b89f651f532312d3'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:17.486440 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-755bda39-dc39-41b5-ae38-e9abbda1d1cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7aa17006bd64559b89f651f532312d3'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:17.493057 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1279/2566] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:34:17 2022] DELETE /identity/v3/users/b7aa17006bd64559b89f651f532312d3 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:17.498554 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7bc10-3111-4213-8c37-08185b9602a3 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:17.499242 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7bc10-3111-4213-8c37-08185b9602a3 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:17.499354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7bc10-3111-4213-8c37-08185b9602a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:17.523770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-44f7bc10-3111-4213-8c37-08185b9602a3 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:34:17.524994 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-44f7bc10-3111-4213-8c37-08185b9602a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:17.590508 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1288/2567] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:34:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:34:17.600875 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755bda39-dc39-41b5-ae38-e9abbda1d1cf admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:17.603025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-755bda39-dc39-41b5-ae38-e9abbda1d1cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:17.665719 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755bda39-dc39-41b5-ae38-e9abbda1d1cf admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:17.670470 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2d92f-7404-4696-b517-fa8f02cf2a53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:17.671074 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2d92f-7404-4696-b517-fa8f02cf2a53 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:17.671350 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2d92f-7404-4696-b517-fa8f02cf2a53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:17.671615 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2d92f-7404-4696-b517-fa8f02cf2a53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:17.684666 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2d92f-7404-4696-b517-fa8f02cf2a53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:17.686541 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2d92f-7404-4696-b517-fa8f02cf2a53 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:17.701052 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1280/2568] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:34:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:34:17.985876 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7bc10-3111-4213-8c37-08185b9602a3 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:17.988547 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-44f7bc10-3111-4213-8c37-08185b9602a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:18.003134 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7bc10-3111-4213-8c37-08185b9602a3 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:18.007863 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405870a3-be76-46df-b458-47e3d088cdb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:18.008531 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405870a3-be76-46df-b458-47e3d088cdb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:18.008813 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405870a3-be76-46df-b458-47e3d088cdb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:18.009130 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405870a3-be76-46df-b458-47e3d088cdb9 admin admin] PATH_INFO: `/v3/projects/c285b3c892ec494dac9fc255c4178606` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:18.010242 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405870a3-be76-46df-b458-47e3d088cdb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c285b3c892ec494dac9fc255c4178606)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:18.012395 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405870a3-be76-46df-b458-47e3d088cdb9 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:18.076521 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-405870a3-be76-46df-b458-47e3d088cdb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c285b3c892ec494dac9fc255c4178606', 'request_id': 'req-405870a3-be76-46df-b458-47e3d088cdb9'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:18.083892 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-405870a3-be76-46df-b458-47e3d088cdb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c285b3c892ec494dac9fc255c4178606', 'request_id': 'req-405870a3-be76-46df-b458-47e3d088cdb9'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:18.092834 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-405870a3-be76-46df-b458-47e3d088cdb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c285b3c892ec494dac9fc255c4178606', 'request_id': 'req-405870a3-be76-46df-b458-47e3d088cdb9'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:18.103716 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-405870a3-be76-46df-b458-47e3d088cdb9 admin admin] The token cache is being invalidate because project c285b3c892ec494dac9fc255c4178606 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:34:18.103956 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-405870a3-be76-46df-b458-47e3d088cdb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:18.104575 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-405870a3-be76-46df-b458-47e3d088cdb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:18.106068 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1289/2569] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:34:17 2022] DELETE /identity/v3/projects/c285b3c892ec494dac9fc255c4178606 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:18.110199 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2d92f-7404-4696-b517-fa8f02cf2a53 service neutron] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:18.119861 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2d92f-7404-4696-b517-fa8f02cf2a53 service neutron] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:18.159906 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405870a3-be76-46df-b458-47e3d088cdb9 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:18.161543 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-405870a3-be76-46df-b458-47e3d088cdb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:18.187116 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd3ad1-4fec-4209-ac1b-52139490047d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:18.187736 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd3ad1-4fec-4209-ac1b-52139490047d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:18.188193 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd3ad1-4fec-4209-ac1b-52139490047d admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:18.188485 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd3ad1-4fec-4209-ac1b-52139490047d admin admin] PATH_INFO: `/v3/users/cf7f923c92014fda85af3e30cb36d38a` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:18.189683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd3ad1-4fec-4209-ac1b-52139490047d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf7f923c92014fda85af3e30cb36d38a)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:18.191616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd3ad1-4fec-4209-ac1b-52139490047d admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:18.208418 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:18.237988 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405870a3-be76-46df-b458-47e3d088cdb9 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:18.243090 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a797c27-361a-4d26-93f7-438a5762ce4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:18.243788 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a797c27-361a-4d26-93f7-438a5762ce4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:18.244107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a797c27-361a-4d26-93f7-438a5762ce4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:18.244406 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a797c27-361a-4d26-93f7-438a5762ce4b admin admin] PATH_INFO: `/v3/users/c64118e6e66e445e8fd0b17dfc80c702` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:18.245678 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a797c27-361a-4d26-93f7-438a5762ce4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c64118e6e66e445e8fd0b17dfc80c702)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:18.247777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a797c27-361a-4d26-93f7-438a5762ce4b admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:18.249376 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-58bd3ad1-4fec-4209-ac1b-52139490047d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf7f923c92014fda85af3e30cb36d38a'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:18.256920 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-58bd3ad1-4fec-4209-ac1b-52139490047d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf7f923c92014fda85af3e30cb36d38a'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:18.258130 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:18.272206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-58bd3ad1-4fec-4209-ac1b-52139490047d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf7f923c92014fda85af3e30cb36d38a'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:18.273149 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-58bd3ad1-4fec-4209-ac1b-52139490047d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf7f923c92014fda85af3e30cb36d38a'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:18.281406 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-58bd3ad1-4fec-4209-ac1b-52139490047d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf7f923c92014fda85af3e30cb36d38a'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:18.287546 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1281/2570] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:34:18 2022] DELETE /identity/v3/users/cf7f923c92014fda85af3e30cb36d38a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:18.297031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-9a797c27-361a-4d26-93f7-438a5762ce4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c64118e6e66e445e8fd0b17dfc80c702'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:18.297695 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-9a797c27-361a-4d26-93f7-438a5762ce4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c64118e6e66e445e8fd0b17dfc80c702'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:18.304966 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-9a797c27-361a-4d26-93f7-438a5762ce4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c64118e6e66e445e8fd0b17dfc80c702'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:18.318453 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-9a797c27-361a-4d26-93f7-438a5762ce4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c64118e6e66e445e8fd0b17dfc80c702'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:18.326436 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-9a797c27-361a-4d26-93f7-438a5762ce4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c64118e6e66e445e8fd0b17dfc80c702'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:18.332787 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1290/2571] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:34:18 2022] DELETE /identity/v3/users/c64118e6e66e445e8fd0b17dfc80c702 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:18.337018 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2bc96c-b4a1-412b-93dd-cd8533ce6d28 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:18.337578 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2bc96c-b4a1-412b-93dd-cd8533ce6d28 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:18.337578 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2bc96c-b4a1-412b-93dd-cd8533ce6d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:18.358847 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-9c2bc96c-b4a1-412b-93dd-cd8533ce6d28 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:34:18.360342 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-9c2bc96c-b4a1-412b-93dd-cd8533ce6d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:18.421919 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1282/2572] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:34:18.432020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a797c27-361a-4d26-93f7-438a5762ce4b admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:18.433742 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-9a797c27-361a-4d26-93f7-438a5762ce4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:18.500836 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a797c27-361a-4d26-93f7-438a5762ce4b admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:18.504904 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd0c40-2da5-4e5a-9c3d-09f8d5fa13fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:18.505542 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd0c40-2da5-4e5a-9c3d-09f8d5fa13fb service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:18.505823 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd0c40-2da5-4e5a-9c3d-09f8d5fa13fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:18.506096 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd0c40-2da5-4e5a-9c3d-09f8d5fa13fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:18.516475 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dd0c40-2da5-4e5a-9c3d-09f8d5fa13fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:18.517922 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dd0c40-2da5-4e5a-9c3d-09f8d5fa13fb service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:18.527493 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1291/2573] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:34:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:34:18.599941 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2bc96c-b4a1-412b-93dd-cd8533ce6d28 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:18.601979 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-9c2bc96c-b4a1-412b-93dd-cd8533ce6d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:18.614753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2bc96c-b4a1-412b-93dd-cd8533ce6d28 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:18.620648 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8404268-2e7d-40fa-b034-a2bacfb33234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:18.621550 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8404268-2e7d-40fa-b034-a2bacfb33234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:18.621865 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8404268-2e7d-40fa-b034-a2bacfb33234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:18.622111 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8404268-2e7d-40fa-b034-a2bacfb33234 admin admin] PATH_INFO: `/v3/projects/bed0b66099034c1c84b3477e118e58b2` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:18.623677 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8404268-2e7d-40fa-b034-a2bacfb33234 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bed0b66099034c1c84b3477e118e58b2)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:18.625680 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8404268-2e7d-40fa-b034-a2bacfb33234 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:18.678199 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d8404268-2e7d-40fa-b034-a2bacfb33234 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bed0b66099034c1c84b3477e118e58b2', 'request_id': 'req-d8404268-2e7d-40fa-b034-a2bacfb33234'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:18.684592 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d8404268-2e7d-40fa-b034-a2bacfb33234 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bed0b66099034c1c84b3477e118e58b2', 'request_id': 'req-d8404268-2e7d-40fa-b034-a2bacfb33234'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:18.693579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d8404268-2e7d-40fa-b034-a2bacfb33234 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bed0b66099034c1c84b3477e118e58b2', 'request_id': 'req-d8404268-2e7d-40fa-b034-a2bacfb33234'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:18.705655 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d8404268-2e7d-40fa-b034-a2bacfb33234 admin admin] The token cache is being invalidate because project bed0b66099034c1c84b3477e118e58b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:34:18.705972 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d8404268-2e7d-40fa-b034-a2bacfb33234 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:18.706729 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d8404268-2e7d-40fa-b034-a2bacfb33234 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:18.710023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1283/2574] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:34:18 2022] DELETE /identity/v3/projects/bed0b66099034c1c84b3477e118e58b2 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:18.714783 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa08d7-587d-43ac-870a-2892e5c25559 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:18.715084 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa08d7-587d-43ac-870a-2892e5c25559 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:18.715376 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa08d7-587d-43ac-870a-2892e5c25559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:18.741455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-76fa08d7-587d-43ac-870a-2892e5c25559 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:34:18.742734 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-76fa08d7-587d-43ac-870a-2892e5c25559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:18.797523 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:18.809098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1292/2575] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:34:18.817357 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d83f8fd-b8d2-469c-95f7-fec69bbdd4c6 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:18.817797 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d83f8fd-b8d2-469c-95f7-fec69bbdd4c6 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:18.818239 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d83f8fd-b8d2-469c-95f7-fec69bbdd4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:18.839013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-3d83f8fd-b8d2-469c-95f7-fec69bbdd4c6 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:34:18.840403 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-3d83f8fd-b8d2-469c-95f7-fec69bbdd4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:18.851038 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1284/2576] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:34:18.855159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d83f8fd-b8d2-469c-95f7-fec69bbdd4c6 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:18.855601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa08d7-587d-43ac-870a-2892e5c25559 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:18.857106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-76fa08d7-587d-43ac-870a-2892e5c25559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:18.866504 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d83f8fd-b8d2-469c-95f7-fec69bbdd4c6 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:18.870439 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa08d7-587d-43ac-870a-2892e5c25559 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:18.872626 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a43f3-1c14-4fde-8ddd-89b485b29ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:18.873325 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a43f3-1c14-4fde-8ddd-89b485b29ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:18.873741 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a43f3-1c14-4fde-8ddd-89b485b29ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:18.874098 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a43f3-1c14-4fde-8ddd-89b485b29ec0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:18.874805 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a43f3-1c14-4fde-8ddd-89b485b29ec0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:34:18.875175 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a43f3-1c14-4fde-8ddd-89b485b29ec0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:18.875395 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c016d0-bb7b-4eef-b4a1-2f33eff15eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:18.876159 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c016d0-bb7b-4eef-b4a1-2f33eff15eea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:18.876494 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c016d0-bb7b-4eef-b4a1-2f33eff15eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:18.876871 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c016d0-bb7b-4eef-b4a1-2f33eff15eea admin admin] PATH_INFO: `/v3/projects/98c1de443dd14892a089e9f9196178a4` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:18.877176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a43f3-1c14-4fde-8ddd-89b485b29ec0 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:18.878109 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c016d0-bb7b-4eef-b4a1-2f33eff15eea admin admin] RBAC: Authorizing `identity:delete_project(project_id=98c1de443dd14892a089e9f9196178a4)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:18.880147 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c016d0-bb7b-4eef-b4a1-2f33eff15eea admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:18.894527 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1285/2577] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:34:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:34:18.895508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a43f3-1c14-4fde-8ddd-89b485b29ec0 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:18.906138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a43f3-1c14-4fde-8ddd-89b485b29ec0 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:18.911459 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138cc258-b56c-4d84-bc05-0aebf32905d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:18.912152 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138cc258-b56c-4d84-bc05-0aebf32905d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:18.912406 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138cc258-b56c-4d84-bc05-0aebf32905d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:18.912678 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138cc258-b56c-4d84-bc05-0aebf32905d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:18.913293 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138cc258-b56c-4d84-bc05-0aebf32905d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:18.915196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138cc258-b56c-4d84-bc05-0aebf32905d1 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:18.931132 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-22c016d0-bb7b-4eef-b4a1-2f33eff15eea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98c1de443dd14892a089e9f9196178a4', 'request_id': 'req-22c016d0-bb7b-4eef-b4a1-2f33eff15eea'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:18.938833 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-22c016d0-bb7b-4eef-b4a1-2f33eff15eea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98c1de443dd14892a089e9f9196178a4', 'request_id': 'req-22c016d0-bb7b-4eef-b4a1-2f33eff15eea'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:18.945948 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1286/2578] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:34:18 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:34:18.946538 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-22c016d0-bb7b-4eef-b4a1-2f33eff15eea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98c1de443dd14892a089e9f9196178a4', 'request_id': 'req-22c016d0-bb7b-4eef-b4a1-2f33eff15eea'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:18.950442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138cc258-b56c-4d84-bc05-0aebf32905d1 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:18.957979 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-22c016d0-bb7b-4eef-b4a1-2f33eff15eea admin admin] The token cache is being invalidate because project 98c1de443dd14892a089e9f9196178a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:34:18.958276 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-22c016d0-bb7b-4eef-b4a1-2f33eff15eea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:18.959611 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-22c016d0-bb7b-4eef-b4a1-2f33eff15eea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:18.961369 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1293/2579] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:34:18 2022] DELETE /identity/v3/projects/98c1de443dd14892a089e9f9196178a4 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:18.965401 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:18.967142 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c463a72-14fd-4395-abc8-e2a318a719c9 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:18.967420 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c463a72-14fd-4395-abc8-e2a318a719c9 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:18.967741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c463a72-14fd-4395-abc8-e2a318a719c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:18.988509 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-7c463a72-14fd-4395-abc8-e2a318a719c9 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:34:18.989889 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-7c463a72-14fd-4395-abc8-e2a318a719c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:19.020714 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138cc258-b56c-4d84-bc05-0aebf32905d1 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:19.026028 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb7be9-9883-4413-9cf7-25783ec16d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:19.026652 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb7be9-9883-4413-9cf7-25783ec16d7e admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:19.026917 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb7be9-9883-4413-9cf7-25783ec16d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:19.027182 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb7be9-9883-4413-9cf7-25783ec16d7e admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:19.027813 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb7be9-9883-4413-9cf7-25783ec16d7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:19.029850 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb7be9-9883-4413-9cf7-25783ec16d7e admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:19.051327 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1294/2580] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:34:19.053864 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1287/2581] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:34:18 2022] POST /identity/v3/users => generated 283 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:34:19.057801 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c463a72-14fd-4395-abc8-e2a318a719c9 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:19.059150 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-7c463a72-14fd-4395-abc8-e2a318a719c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:19.061095 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2ac8f-1706-4013-aad6-e41ea9fab830 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:19.061404 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2ac8f-1706-4013-aad6-e41ea9fab830 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:19.061680 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2ac8f-1706-4013-aad6-e41ea9fab830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:19.071562 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c463a72-14fd-4395-abc8-e2a318a719c9 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:19.076544 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac652b-3b6e-47e8-b310-1638fe68a8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:19.077160 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac652b-3b6e-47e8-b310-1638fe68a8eb admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:19.077604 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac652b-3b6e-47e8-b310-1638fe68a8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:19.077879 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac652b-3b6e-47e8-b310-1638fe68a8eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:19.078529 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ac652b-3b6e-47e8-b310-1638fe68a8eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:19.080233 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ac652b-3b6e-47e8-b310-1638fe68a8eb admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:19.082905 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-0cd2ac8f-1706-4013-aad6-e41ea9fab830 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:34:19.084189 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-0cd2ac8f-1706-4013-aad6-e41ea9fab830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:19.090905 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1295/2582] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:34:19.092685 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1288/2583] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:34:19.098152 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac652b-3b6e-47e8-b310-1638fe68a8eb admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:19.098315 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2ac8f-1706-4013-aad6-e41ea9fab830 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:19.109569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2ac8f-1706-4013-aad6-e41ea9fab830 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:19.112238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac652b-3b6e-47e8-b310-1638fe68a8eb admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:19.117995 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ff312-24b1-4cbe-a6b4-77462c21125b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:19.118611 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ff312-24b1-4cbe-a6b4-77462c21125b admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:19.118792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ff312-24b1-4cbe-a6b4-77462c21125b admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:19.119080 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ff312-24b1-4cbe-a6b4-77462c21125b admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:19.119754 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ff312-24b1-4cbe-a6b4-77462c21125b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:19.122106 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ff312-24b1-4cbe-a6b4-77462c21125b admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:19.124160 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8fceaa-7026-4567-bd5a-1d21fab10a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:19.127569 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fceaa-7026-4567-bd5a-1d21fab10a7b admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:19.127569 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fceaa-7026-4567-bd5a-1d21fab10a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:19.127569 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fceaa-7026-4567-bd5a-1d21fab10a7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:19.127569 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8fceaa-7026-4567-bd5a-1d21fab10a7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:34:19.127569 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8fceaa-7026-4567-bd5a-1d21fab10a7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:19.129220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8fceaa-7026-4567-bd5a-1d21fab10a7b admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:19.133836 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1289/2584] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:34:19.134675 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ff312-24b1-4cbe-a6b4-77462c21125b admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:19.136400 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-066ff312-24b1-4cbe-a6b4-77462c21125b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:19.143245 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1296/2585] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:34:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:34:19.144126 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8fceaa-7026-4567-bd5a-1d21fab10a7b admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:19.151524 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ff312-24b1-4cbe-a6b4-77462c21125b admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:19.155603 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8fceaa-7026-4567-bd5a-1d21fab10a7b admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:19.156776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea310b-27d3-4250-a3ba-818665e9476c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:19.157547 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea310b-27d3-4250-a3ba-818665e9476c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:19.157885 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea310b-27d3-4250-a3ba-818665e9476c admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:19.158160 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea310b-27d3-4250-a3ba-818665e9476c admin admin] PATH_INFO: `/v3/users/57c565869ac14ef0bf6e5f42273523e8` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:19.159413 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea310b-27d3-4250-a3ba-818665e9476c admin admin] RBAC: Authorizing `identity:delete_user(user_id=57c565869ac14ef0bf6e5f42273523e8)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:19.160272 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3731524-2e1b-40fc-8bf3-8fa5d9b14d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:19.160916 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3731524-2e1b-40fc-8bf3-8fa5d9b14d7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:19.161189 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3731524-2e1b-40fc-8bf3-8fa5d9b14d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:19.161457 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3731524-2e1b-40fc-8bf3-8fa5d9b14d7f admin admin] PATH_INFO: `/v3/projects/f8c314d0382943438b2182f2800101e1/users/760985d5610c4b61a61cac2dbf9449d2/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:19.161553 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea310b-27d3-4250-a3ba-818665e9476c admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:19.177155 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3731524-2e1b-40fc-8bf3-8fa5d9b14d7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8c314d0382943438b2182f2800101e1, user_id=760985d5610c4b61a61cac2dbf9449d2, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:19.179780 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3731524-2e1b-40fc-8bf3-8fa5d9b14d7f admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:19.193708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1297/2586] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:34:19 2022] PUT /identity/v3/projects/f8c314d0382943438b2182f2800101e1/users/760985d5610c4b61a61cac2dbf9449d2/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:19.194552 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3731524-2e1b-40fc-8bf3-8fa5d9b14d7f admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:19.221781 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-59ea310b-27d3-4250-a3ba-818665e9476c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57c565869ac14ef0bf6e5f42273523e8'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:19.233097 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-59ea310b-27d3-4250-a3ba-818665e9476c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57c565869ac14ef0bf6e5f42273523e8'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:19.248816 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:19.254607 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-59ea310b-27d3-4250-a3ba-818665e9476c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57c565869ac14ef0bf6e5f42273523e8'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:19.255760 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-59ea310b-27d3-4250-a3ba-818665e9476c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57c565869ac14ef0bf6e5f42273523e8'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:19.266929 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-59ea310b-27d3-4250-a3ba-818665e9476c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57c565869ac14ef0bf6e5f42273523e8'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:19.273248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1290/2587] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:34:19 2022] DELETE /identity/v3/users/57c565869ac14ef0bf6e5f42273523e8 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:19.274496 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177e177-f8f0-4059-932d-ffced48a6cb4 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:19.274781 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177e177-f8f0-4059-932d-ffced48a6cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:19.275057 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177e177-f8f0-4059-932d-ffced48a6cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:19.282029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3731524-2e1b-40fc-8bf3-8fa5d9b14d7f admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:19.303154 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-6177e177-f8f0-4059-932d-ffced48a6cb4 None None] MFA Rules not processed for user `760985d5610c4b61a61cac2dbf9449d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:34:19.304487 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-6177e177-f8f0-4059-932d-ffced48a6cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:19.340012 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67b22d-58f3-4740-9320-557bbdc9ef43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:19.340627 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67b22d-58f3-4740-9320-557bbdc9ef43 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:19.340908 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67b22d-58f3-4740-9320-557bbdc9ef43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:19.341235 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67b22d-58f3-4740-9320-557bbdc9ef43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:19.342118 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67b22d-58f3-4740-9320-557bbdc9ef43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:19.344072 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67b22d-58f3-4740-9320-557bbdc9ef43 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:19.350128 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:19.376923 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1291/2588] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:34:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:34:19.377103 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1298/2589] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:34:19 2022] POST /identity/v3/projects => generated 377 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:34:19.377196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7926f-f869-498a-93be-f4e5dc52a351 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:19.377196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7926f-f869-498a-93be-f4e5dc52a351 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:19.377196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7926f-f869-498a-93be-f4e5dc52a351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:19.380899 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67b22d-58f3-4740-9320-557bbdc9ef43 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:19.383272 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-be67b22d-58f3-4740-9320-557bbdc9ef43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:19.395445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-ccd7926f-f869-498a-93be-f4e5dc52a351 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:34:19.396824 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-ccd7926f-f869-498a-93be-f4e5dc52a351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:19.452873 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67b22d-58f3-4740-9320-557bbdc9ef43 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:19.458997 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194f16e-3b0f-4f8d-b74e-7735e8d192d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:19.458997 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194f16e-3b0f-4f8d-b74e-7735e8d192d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:19.459456 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194f16e-3b0f-4f8d-b74e-7735e8d192d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:19.459934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194f16e-3b0f-4f8d-b74e-7735e8d192d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:19.460926 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9194f16e-3b0f-4f8d-b74e-7735e8d192d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:19.462904 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9194f16e-3b0f-4f8d-b74e-7735e8d192d5 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:19.465336 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1292/2590] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:34:19.465336 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7926f-f869-498a-93be-f4e5dc52a351 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:19.467187 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-ccd7926f-f869-498a-93be-f4e5dc52a351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:19.496319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1299/2591] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:34:19 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:34:19.496319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194f16e-3b0f-4f8d-b74e-7735e8d192d5 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:19.497829 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-9194f16e-3b0f-4f8d-b74e-7735e8d192d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:19.529209 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7926f-f869-498a-93be-f4e5dc52a351 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:19.533499 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0ec15-2e73-404a-8314-dae5434d4eeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:19.534350 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0ec15-2e73-404a-8314-dae5434d4eeb service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:19.535468 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0ec15-2e73-404a-8314-dae5434d4eeb service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:19.535765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0ec15-2e73-404a-8314-dae5434d4eeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:19.558402 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194f16e-3b0f-4f8d-b74e-7735e8d192d5 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:19.562844 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a8663-fb52-43cd-b629-9f5d2a4517f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:19.563737 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a8663-fb52-43cd-b629-9f5d2a4517f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:19.564058 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a8663-fb52-43cd-b629-9f5d2a4517f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:19.564303 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a8663-fb52-43cd-b629-9f5d2a4517f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:19.574141 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a8663-fb52-43cd-b629-9f5d2a4517f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:19.575639 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a8663-fb52-43cd-b629-9f5d2a4517f0 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:19.581311 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:19.588332 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1300/2592] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:34:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:34:19.589271 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a8663-fb52-43cd-b629-9f5d2a4517f0 service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:19.598227 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0ec15-2e73-404a-8314-dae5434d4eeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:19.599694 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0ec15-2e73-404a-8314-dae5434d4eeb service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:19.600173 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a8663-fb52-43cd-b629-9f5d2a4517f0 service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:19.607210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75590a47-0bbe-48fa-a38b-3ce46f4206c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:19.607950 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75590a47-0bbe-48fa-a38b-3ce46f4206c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:19.608244 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75590a47-0bbe-48fa-a38b-3ce46f4206c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:19.608643 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75590a47-0bbe-48fa-a38b-3ce46f4206c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:19.609313 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75590a47-0bbe-48fa-a38b-3ce46f4206c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:19.610954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75590a47-0bbe-48fa-a38b-3ce46f4206c0 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:19.611385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1293/2593] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:34:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:34:19.623428 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1301/2594] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:34:19.627765 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0ec15-2e73-404a-8314-dae5434d4eeb service cinder] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:19.639428 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0ec15-2e73-404a-8314-dae5434d4eeb service cinder] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:19.644559 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9414388-9f00-4799-b7b8-17c7bddae112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:19.645324 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9414388-9f00-4799-b7b8-17c7bddae112 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:19.645613 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9414388-9f00-4799-b7b8-17c7bddae112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:19.645920 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9414388-9f00-4799-b7b8-17c7bddae112 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:19.646616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9414388-9f00-4799-b7b8-17c7bddae112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:19.648347 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9414388-9f00-4799-b7b8-17c7bddae112 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:19.662880 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1294/2595] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:34:19.665367 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75590a47-0bbe-48fa-a38b-3ce46f4206c0 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:19.678050 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75590a47-0bbe-48fa-a38b-3ce46f4206c0 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:19.684722 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408f1ab-852a-4713-a266-eae9b7feaace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:19.686326 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408f1ab-852a-4713-a266-eae9b7feaace admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:19.686725 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408f1ab-852a-4713-a266-eae9b7feaace admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:19.687003 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408f1ab-852a-4713-a266-eae9b7feaace admin admin] PATH_INFO: `/v3/projects/7d935a9f23f64970ab825f32a069fdd3/users/95fb255f4c4342daa13314fe74172887/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:19.708202 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408f1ab-852a-4713-a266-eae9b7feaace admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d935a9f23f64970ab825f32a069fdd3, user_id=95fb255f4c4342daa13314fe74172887, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:19.713204 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408f1ab-852a-4713-a266-eae9b7feaace admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:19.731082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1302/2596] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:34:19 2022] PUT /identity/v3/projects/7d935a9f23f64970ab825f32a069fdd3/users/95fb255f4c4342daa13314fe74172887/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:19.741893 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534554f-a1d0-42f6-aaaa-292388984651 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:19.741996 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534554f-a1d0-42f6-aaaa-292388984651 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:19.742191 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534554f-a1d0-42f6-aaaa-292388984651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:19.782102 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-7534554f-a1d0-42f6-aaaa-292388984651 None None] MFA Rules not processed for user `95fb255f4c4342daa13314fe74172887`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:34:19.783873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-7534554f-a1d0-42f6-aaaa-292388984651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:19.811287 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:19.870829 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1295/2597] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:34:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:34:19.875666 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408f1ab-852a-4713-a266-eae9b7feaace admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:19.898983 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534554f-a1d0-42f6-aaaa-292388984651 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:19.900803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-7534554f-a1d0-42f6-aaaa-292388984651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:19.991820 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408f1ab-852a-4713-a266-eae9b7feaace admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:20.003318 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534554f-a1d0-42f6-aaaa-292388984651 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:20.005544 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e70120-dece-4667-a31c-75bda32367ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:20.006177 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e70120-dece-4667-a31c-75bda32367ba admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:20.006482 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e70120-dece-4667-a31c-75bda32367ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:20.007437 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e70120-dece-4667-a31c-75bda32367ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:20.007559 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e70120-dece-4667-a31c-75bda32367ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:20.009427 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07cf56-0e45-41db-8dcb-78e612582d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:20.009570 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e70120-dece-4667-a31c-75bda32367ba admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:20.010207 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07cf56-0e45-41db-8dcb-78e612582d1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:20.010551 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07cf56-0e45-41db-8dcb-78e612582d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:20.010883 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07cf56-0e45-41db-8dcb-78e612582d1c admin admin] PATH_INFO: `/v3/projects/cc551179c4fd40ef93d92d1a1c8e11bc` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:20.012236 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07cf56-0e45-41db-8dcb-78e612582d1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc551179c4fd40ef93d92d1a1c8e11bc)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:20.014750 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07cf56-0e45-41db-8dcb-78e612582d1c admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:20.044293 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1303/2598] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:34:19 2022] POST /identity/v3/projects => generated 377 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:34:20.048638 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e70120-dece-4667-a31c-75bda32367ba admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:20.069249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-4d07cf56-0e45-41db-8dcb-78e612582d1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc551179c4fd40ef93d92d1a1c8e11bc', 'request_id': 'req-4d07cf56-0e45-41db-8dcb-78e612582d1c'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:20.075607 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-4d07cf56-0e45-41db-8dcb-78e612582d1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc551179c4fd40ef93d92d1a1c8e11bc', 'request_id': 'req-4d07cf56-0e45-41db-8dcb-78e612582d1c'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:20.084274 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-4d07cf56-0e45-41db-8dcb-78e612582d1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc551179c4fd40ef93d92d1a1c8e11bc', 'request_id': 'req-4d07cf56-0e45-41db-8dcb-78e612582d1c'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:20.095804 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-4d07cf56-0e45-41db-8dcb-78e612582d1c admin admin] The token cache is being invalidate because project cc551179c4fd40ef93d92d1a1c8e11bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:34:20.096127 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-4d07cf56-0e45-41db-8dcb-78e612582d1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:20.098158 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-4d07cf56-0e45-41db-8dcb-78e612582d1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:20.099741 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1296/2599] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:34:19 2022] DELETE /identity/v3/projects/cc551179c4fd40ef93d92d1a1c8e11bc => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:34:20.105085 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101ebcf-6f2a-4942-856e-14b912ce1064 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:20.105377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101ebcf-6f2a-4942-856e-14b912ce1064 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:20.105654 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101ebcf-6f2a-4942-856e-14b912ce1064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:20.116617 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e70120-dece-4667-a31c-75bda32367ba admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:20.125623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-4101ebcf-6f2a-4942-856e-14b912ce1064 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:34:20.126892 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-4101ebcf-6f2a-4942-856e-14b912ce1064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:20.168175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:20.175164 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba08e8c-9d50-4e72-a6f1-689ec0785e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:20.175790 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba08e8c-9d50-4e72-a6f1-689ec0785e33 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:20.176178 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba08e8c-9d50-4e72-a6f1-689ec0785e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:20.176454 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba08e8c-9d50-4e72-a6f1-689ec0785e33 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:20.176756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:20.177488 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba08e8c-9d50-4e72-a6f1-689ec0785e33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:20.179750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba08e8c-9d50-4e72-a6f1-689ec0785e33 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:20.192921 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1297/2600] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:34:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:34:20.202025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06729783-6757-437a-a2ca-35216b595b45 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:20.202189 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1304/2601] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:34:20 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:34:20.205179 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06729783-6757-437a-a2ca-35216b595b45 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:20.205475 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06729783-6757-437a-a2ca-35216b595b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:20.206347 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba08e8c-9d50-4e72-a6f1-689ec0785e33 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:20.208139 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-1ba08e8c-9d50-4e72-a6f1-689ec0785e33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:20.220423 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba08e8c-9d50-4e72-a6f1-689ec0785e33 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:20.225890 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-06729783-6757-437a-a2ca-35216b595b45 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:34:20.226190 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8710ee-4867-49d0-99a4-9191642db6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:20.226720 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8710ee-4867-49d0-99a4-9191642db6b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:20.227034 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8710ee-4867-49d0-99a4-9191642db6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:20.227299 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8710ee-4867-49d0-99a4-9191642db6b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:20.227418 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-06729783-6757-437a-a2ca-35216b595b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:20.228055 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8710ee-4867-49d0-99a4-9191642db6b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:20.229647 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8710ee-4867-49d0-99a4-9191642db6b4 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:20.235785 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1298/2602] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:34:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:34:20.241012 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06729783-6757-437a-a2ca-35216b595b45 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:20.244623 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1305/2603] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:34:20.249108 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8710ee-4867-49d0-99a4-9191642db6b4 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:20.253442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06729783-6757-437a-a2ca-35216b595b45 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:20.259359 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c36176-3e65-4b02-a85a-7a0c07dc67d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:20.259746 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c36176-3e65-4b02-a85a-7a0c07dc67d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:20.260138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c36176-3e65-4b02-a85a-7a0c07dc67d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:20.260338 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c36176-3e65-4b02-a85a-7a0c07dc67d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:20.261072 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c36176-3e65-4b02-a85a-7a0c07dc67d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:34:20.261437 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8710ee-4867-49d0-99a4-9191642db6b4 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:20.261555 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c36176-3e65-4b02-a85a-7a0c07dc67d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:20.263402 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c36176-3e65-4b02-a85a-7a0c07dc67d8 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:20.267365 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78779d2-0e89-448a-9768-c2cfa2ff0bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:20.268227 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78779d2-0e89-448a-9768-c2cfa2ff0bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:20.268547 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78779d2-0e89-448a-9768-c2cfa2ff0bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:20.268939 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78779d2-0e89-448a-9768-c2cfa2ff0bb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:20.269637 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78779d2-0e89-448a-9768-c2cfa2ff0bb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:20.271218 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78779d2-0e89-448a-9768-c2cfa2ff0bb9 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:20.276534 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1299/2604] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:34:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:34:20.282264 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c36176-3e65-4b02-a85a-7a0c07dc67d8 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:20.286198 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1306/2605] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:34:20.287273 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78779d2-0e89-448a-9768-c2cfa2ff0bb9 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:20.294204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c36176-3e65-4b02-a85a-7a0c07dc67d8 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:20.297990 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78779d2-0e89-448a-9768-c2cfa2ff0bb9 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:20.298927 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586a076-6210-461e-8193-3807e923aafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:20.299530 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586a076-6210-461e-8193-3807e923aafc admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:20.299811 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586a076-6210-461e-8193-3807e923aafc admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:20.300115 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586a076-6210-461e-8193-3807e923aafc admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:20.300737 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2586a076-6210-461e-8193-3807e923aafc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:20.302651 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2586a076-6210-461e-8193-3807e923aafc admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:20.303204 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6bc889-829f-48b4-bed1-c33e2f89d4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:20.303965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6bc889-829f-48b4-bed1-c33e2f89d4e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:20.304343 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6bc889-829f-48b4-bed1-c33e2f89d4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:20.304709 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6bc889-829f-48b4-bed1-c33e2f89d4e1 admin admin] PATH_INFO: `/v3/projects/9b661e4a3f9a4c32ad1cb83cae23d40b/users/2ddc39d6eb0c470c9184f16b76f1f3fe/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:20.319992 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6bc889-829f-48b4-bed1-c33e2f89d4e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b661e4a3f9a4c32ad1cb83cae23d40b, user_id=2ddc39d6eb0c470c9184f16b76f1f3fe, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:20.322826 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6bc889-829f-48b4-bed1-c33e2f89d4e1 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:20.332300 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1300/2606] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:34:20 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:34:20.336264 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1307/2607] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:34:20 2022] PUT /identity/v3/projects/9b661e4a3f9a4c32ad1cb83cae23d40b/users/2ddc39d6eb0c470c9184f16b76f1f3fe/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:20.345233 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586a076-6210-461e-8193-3807e923aafc admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:20.347076 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ac134-da09-4722-b8fe-8785b026f3be None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:20.347363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ac134-da09-4722-b8fe-8785b026f3be None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:20.347639 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ac134-da09-4722-b8fe-8785b026f3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:20.376018 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-114ac134-da09-4722-b8fe-8785b026f3be None None] MFA Rules not processed for user `2ddc39d6eb0c470c9184f16b76f1f3fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:34:20.377276 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-114ac134-da09-4722-b8fe-8785b026f3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:20.400044 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586a076-6210-461e-8193-3807e923aafc admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:20.405348 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc89f96-f085-4791-bf84-c30a94bc2c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:20.405969 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc89f96-f085-4791-bf84-c30a94bc2c21 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:20.406251 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc89f96-f085-4791-bf84-c30a94bc2c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:20.406524 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc89f96-f085-4791-bf84-c30a94bc2c21 admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:20.407132 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc89f96-f085-4791-bf84-c30a94bc2c21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:20.409136 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc89f96-f085-4791-bf84-c30a94bc2c21 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:20.436015 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1301/2608] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:34:20 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:34:20.437363 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc89f96-f085-4791-bf84-c30a94bc2c21 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:20.444857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1308/2609] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:34:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:34:20.447316 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc89f96-f085-4791-bf84-c30a94bc2c21 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:20.449101 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ac134-da09-4722-b8fe-8785b026f3be None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:20.457753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4049a-3ef4-4b3f-bd54-1e4b95d5914d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:20.458449 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4049a-3ef4-4b3f-bd54-1e4b95d5914d admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:20.458717 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4049a-3ef4-4b3f-bd54-1e4b95d5914d admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:20.458993 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4049a-3ef4-4b3f-bd54-1e4b95d5914d admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:20.459720 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd4049a-3ef4-4b3f-bd54-1e4b95d5914d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:20.461639 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd4049a-3ef4-4b3f-bd54-1e4b95d5914d admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:20.463038 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ac134-da09-4722-b8fe-8785b026f3be None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:20.468448 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72903100-4ac9-43ca-84ef-8e7913feaefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:20.469063 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72903100-4ac9-43ca-84ef-8e7913feaefc admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:20.469376 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72903100-4ac9-43ca-84ef-8e7913feaefc admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:20.469664 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72903100-4ac9-43ca-84ef-8e7913feaefc admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:20.470298 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72903100-4ac9-43ca-84ef-8e7913feaefc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:20.472228 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72903100-4ac9-43ca-84ef-8e7913feaefc admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:20.473330 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1302/2610] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:34:20.477767 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4049a-3ef4-4b3f-bd54-1e4b95d5914d admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:20.488806 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4049a-3ef4-4b3f-bd54-1e4b95d5914d admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:20.498314 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cec803-ffe3-4bff-8238-97d901e0e3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:20.499039 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cec803-ffe3-4bff-8238-97d901e0e3ec admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:20.499319 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cec803-ffe3-4bff-8238-97d901e0e3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:20.499647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cec803-ffe3-4bff-8238-97d901e0e3ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:20.500390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cec803-ffe3-4bff-8238-97d901e0e3ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:20.502081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cec803-ffe3-4bff-8238-97d901e0e3ec admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:20.510807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1309/2611] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:34:20 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:34:20.511948 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72903100-4ac9-43ca-84ef-8e7913feaefc admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:20.514025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1303/2612] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:34:20.518159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cec803-ffe3-4bff-8238-97d901e0e3ec admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:20.528144 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cec803-ffe3-4bff-8238-97d901e0e3ec admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:20.578330 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72903100-4ac9-43ca-84ef-8e7913feaefc admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:20.583141 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d58180-a1d1-45cf-9fb4-c0eae1daf55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:20.583785 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d58180-a1d1-45cf-9fb4-c0eae1daf55e admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:20.584113 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d58180-a1d1-45cf-9fb4-c0eae1daf55e admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:20.584382 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d58180-a1d1-45cf-9fb4-c0eae1daf55e admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:20.585034 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d58180-a1d1-45cf-9fb4-c0eae1daf55e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:20.586420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1203139e-e331-4ed7-9f13-9b8f828b37fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:20.586966 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d58180-a1d1-45cf-9fb4-c0eae1daf55e admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:20.587087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203139e-e331-4ed7-9f13-9b8f828b37fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:20.587375 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203139e-e331-4ed7-9f13-9b8f828b37fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:20.587630 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203139e-e331-4ed7-9f13-9b8f828b37fa admin admin] PATH_INFO: `/v3/projects/d190cd833ca743c08e903166138f008f/users/c35cfc1e230e4b4593a63e9475fb6747/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:20.603614 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1203139e-e331-4ed7-9f13-9b8f828b37fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d190cd833ca743c08e903166138f008f, user_id=c35cfc1e230e4b4593a63e9475fb6747, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:20.604934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:20.606562 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1203139e-e331-4ed7-9f13-9b8f828b37fa admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:20.614731 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1310/2613] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:34:20 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:34:20.615944 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d58180-a1d1-45cf-9fb4-c0eae1daf55e admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:20.620177 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:20.627190 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1304/2614] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:34:20 2022] PUT /identity/v3/projects/d190cd833ca743c08e903166138f008f/users/c35cfc1e230e4b4593a63e9475fb6747/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:20.627579 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d58180-a1d1-45cf-9fb4-c0eae1daf55e admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:20.638069 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d0205-7dc9-418a-8159-328719df870b None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:20.638637 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d0205-7dc9-418a-8159-328719df870b None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:20.638938 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d0205-7dc9-418a-8159-328719df870b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:20.667490 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-678d0205-7dc9-418a-8159-328719df870b None None] MFA Rules not processed for user `c35cfc1e230e4b4593a63e9475fb6747`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:34:20.668771 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-678d0205-7dc9-418a-8159-328719df870b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:20.688641 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220c0d2-bd8a-42cf-b53c-e65ae1473705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:20.689201 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220c0d2-bd8a-42cf-b53c-e65ae1473705 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:20.689481 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220c0d2-bd8a-42cf-b53c-e65ae1473705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:20.689752 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220c0d2-bd8a-42cf-b53c-e65ae1473705 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:20.690452 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220c0d2-bd8a-42cf-b53c-e65ae1473705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:20.692122 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220c0d2-bd8a-42cf-b53c-e65ae1473705 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:20.702579 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1311/2615] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:34:20.706950 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220c0d2-bd8a-42cf-b53c-e65ae1473705 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:20.717491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220c0d2-bd8a-42cf-b53c-e65ae1473705 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:20.723197 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bffecff-c012-497a-ab25-c05fc7fc174f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:20.723870 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bffecff-c012-497a-ab25-c05fc7fc174f admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:20.724179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bffecff-c012-497a-ab25-c05fc7fc174f admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:20.724458 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bffecff-c012-497a-ab25-c05fc7fc174f admin admin] PATH_INFO: `/v3/projects/e16ef088d0934313a3d704d4ad2276e3/users/d9687acf6a55401a95677fb4b86bcb1b/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:20.737508 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1305/2616] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:34:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:34:20.742317 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d0205-7dc9-418a-8159-328719df870b None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:20.742720 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bffecff-c012-497a-ab25-c05fc7fc174f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e16ef088d0934313a3d704d4ad2276e3, user_id=d9687acf6a55401a95677fb4b86bcb1b, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:20.745539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bffecff-c012-497a-ab25-c05fc7fc174f admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:20.754722 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d0205-7dc9-418a-8159-328719df870b None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:20.760285 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1312/2617] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:34:20 2022] PUT /identity/v3/projects/e16ef088d0934313a3d704d4ad2276e3/users/d9687acf6a55401a95677fb4b86bcb1b/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:34:20.764854 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bffecff-c012-497a-ab25-c05fc7fc174f admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:20.814705 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab65fe-920e-42a4-9940-7337815de2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:20.815355 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab65fe-920e-42a4-9940-7337815de2e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:20.815635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab65fe-920e-42a4-9940-7337815de2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:20.815906 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab65fe-920e-42a4-9940-7337815de2e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:20.816587 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab65fe-920e-42a4-9940-7337815de2e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:20.818538 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab65fe-920e-42a4-9940-7337815de2e2 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:20.828535 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bffecff-c012-497a-ab25-c05fc7fc174f admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:20.833323 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c20f3-0c23-4b50-af44-19250ab63d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:20.833930 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c20f3-0c23-4b50-af44-19250ab63d3c admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:20.834208 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c20f3-0c23-4b50-af44-19250ab63d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:20.834481 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c20f3-0c23-4b50-af44-19250ab63d3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:20.835127 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c20f3-0c23-4b50-af44-19250ab63d3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:20.836942 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c20f3-0c23-4b50-af44-19250ab63d3c admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:20.847504 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1313/2618] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:34:20.848930 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1306/2619] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:34:20 2022] POST /identity/v3/projects => generated 371 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:34:20.852321 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c20f3-0c23-4b50-af44-19250ab63d3c admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:20.853025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab65fe-920e-42a4-9940-7337815de2e2 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:20.868520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c20f3-0c23-4b50-af44-19250ab63d3c admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:20.931054 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab65fe-920e-42a4-9940-7337815de2e2 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:20.934311 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef7c9c-c3e3-46ea-849b-1fd3da030205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:20.935199 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef7c9c-c3e3-46ea-849b-1fd3da030205 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:20.935480 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef7c9c-c3e3-46ea-849b-1fd3da030205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:20.935750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef7c9c-c3e3-46ea-849b-1fd3da030205 admin admin] PATH_INFO: `/v3/projects/e16ef088d0934313a3d704d4ad2276e3/users/d9687acf6a55401a95677fb4b86bcb1b/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:20.935944 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762d113-e5c6-488e-9b4d-3c7a4abca4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:20.936633 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762d113-e5c6-488e-9b4d-3c7a4abca4fc admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:20.936932 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762d113-e5c6-488e-9b4d-3c7a4abca4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:20.937217 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762d113-e5c6-488e-9b4d-3c7a4abca4fc admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:20.937807 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef7c9c-c3e3-46ea-849b-1fd3da030205 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e16ef088d0934313a3d704d4ad2276e3, user_id=d9687acf6a55401a95677fb4b86bcb1b, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:20.937944 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762d113-e5c6-488e-9b4d-3c7a4abca4fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:20.939991 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762d113-e5c6-488e-9b4d-3c7a4abca4fc admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:20.940520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef7c9c-c3e3-46ea-849b-1fd3da030205 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:20.946830 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:20.952268 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1314/2620] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:34:20 2022] PUT /identity/v3/projects/e16ef088d0934313a3d704d4ad2276e3/users/d9687acf6a55401a95677fb4b86bcb1b/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:20.967422 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1307/2621] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:34:20 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:34:20.967630 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d74c8-f1cb-4838-ad82-c0c5187c1f96 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:20.967630 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d74c8-f1cb-4838-ad82-c0c5187c1f96 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:20.967630 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d74c8-f1cb-4838-ad82-c0c5187c1f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:20.968706 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762d113-e5c6-488e-9b4d-3c7a4abca4fc admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:20.974911 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:20.979810 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762d113-e5c6-488e-9b4d-3c7a4abca4fc admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:20.998314 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-4b9d74c8-f1cb-4838-ad82-c0c5187c1f96 None None] MFA Rules not processed for user `d9687acf6a55401a95677fb4b86bcb1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:34:20.999766 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-4b9d74c8-f1cb-4838-ad82-c0c5187c1f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:21.060503 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4304f873-34bb-463e-a9b3-b0ec4d7acbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:21.061383 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304f873-34bb-463e-a9b3-b0ec4d7acbad admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:21.062146 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304f873-34bb-463e-a9b3-b0ec4d7acbad admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:21.062480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304f873-34bb-463e-a9b3-b0ec4d7acbad admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:21.063464 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4304f873-34bb-463e-a9b3-b0ec4d7acbad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:21.066091 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4304f873-34bb-463e-a9b3-b0ec4d7acbad admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:21.080492 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1308/2622] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:34:21.084888 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4304f873-34bb-463e-a9b3-b0ec4d7acbad admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:21.091594 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1315/2623] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:34:20 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:34:21.097658 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4304f873-34bb-463e-a9b3-b0ec4d7acbad admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:21.103575 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5cfea4-f239-41d7-9c22-3112db1af00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:21.104434 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5cfea4-f239-41d7-9c22-3112db1af00a admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:21.104771 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5cfea4-f239-41d7-9c22-3112db1af00a admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:21.105115 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5cfea4-f239-41d7-9c22-3112db1af00a admin admin] PATH_INFO: `/v3/projects/f1929dde909144c38aa344ba9efb316f/users/69046982b02b44eeba95903048ea73eb/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:21.105293 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d74c8-f1cb-4838-ad82-c0c5187c1f96 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:21.107029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-4b9d74c8-f1cb-4838-ad82-c0c5187c1f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:21.127279 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5cfea4-f239-41d7-9c22-3112db1af00a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1929dde909144c38aa344ba9efb316f, user_id=69046982b02b44eeba95903048ea73eb, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:21.130544 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5cfea4-f239-41d7-9c22-3112db1af00a admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:21.152909 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1309/2624] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:34:21 2022] PUT /identity/v3/projects/f1929dde909144c38aa344ba9efb316f/users/69046982b02b44eeba95903048ea73eb/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:21.154127 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5cfea4-f239-41d7-9c22-3112db1af00a admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:21.204777 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d74c8-f1cb-4838-ad82-c0c5187c1f96 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:21.248521 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5cfea4-f239-41d7-9c22-3112db1af00a admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:21.256956 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c794731-aa78-4032-8909-af898456eeea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:21.257724 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c794731-aa78-4032-8909-af898456eeea admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:21.258005 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c794731-aa78-4032-8909-af898456eeea admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:21.258273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c794731-aa78-4032-8909-af898456eeea admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:21.259051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c794731-aa78-4032-8909-af898456eeea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:21.260806 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c794731-aa78-4032-8909-af898456eeea admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:21.272067 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69baf79-3e0d-42c6-a35c-f8fd7d314707 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:21.273489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69baf79-3e0d-42c6-a35c-f8fd7d314707 service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:21.273693 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69baf79-3e0d-42c6-a35c-f8fd7d314707 service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:21.273962 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69baf79-3e0d-42c6-a35c-f8fd7d314707 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:21.274619 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1310/2625] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:34:21.275835 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-f69baf79-3e0d-42c6-a35c-f8fd7d314707 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:21.280020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c794731-aa78-4032-8909-af898456eeea admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:21.294026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c794731-aa78-4032-8909-af898456eeea admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:21.301646 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7693bd-6e69-4603-a55f-f9deb2c75ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:21.302382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7693bd-6e69-4603-a55f-f9deb2c75ffe admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:21.303299 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7693bd-6e69-4603-a55f-f9deb2c75ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:21.303727 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7693bd-6e69-4603-a55f-f9deb2c75ffe admin admin] PATH_INFO: `/v3/projects/f1929dde909144c38aa344ba9efb316f/users/69046982b02b44eeba95903048ea73eb/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:21.306048 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7693bd-6e69-4603-a55f-f9deb2c75ffe admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1929dde909144c38aa344ba9efb316f, user_id=69046982b02b44eeba95903048ea73eb, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:21.309194 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7693bd-6e69-4603-a55f-f9deb2c75ffe admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:21.323745 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1311/2626] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:34:21 2022] PUT /identity/v3/projects/f1929dde909144c38aa344ba9efb316f/users/69046982b02b44eeba95903048ea73eb/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:34:21.334036 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79f660-29f8-4c35-8f0d-04ddf41344c5 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:21.334327 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79f660-29f8-4c35-8f0d-04ddf41344c5 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:21.334600 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79f660-29f8-4c35-8f0d-04ddf41344c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:21.366656 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-cd79f660-29f8-4c35-8f0d-04ddf41344c5 None None] MFA Rules not processed for user `69046982b02b44eeba95903048ea73eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:34:21.367986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-cd79f660-29f8-4c35-8f0d-04ddf41344c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:21.377713 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:21.382185 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69baf79-3e0d-42c6-a35c-f8fd7d314707 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:21.384782 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69baf79-3e0d-42c6-a35c-f8fd7d314707 service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:21.398654 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1316/2627] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:34:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:34:21.435647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:21.475593 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1312/2628] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:34:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:34:21.561497 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69baf79-3e0d-42c6-a35c-f8fd7d314707 service cinder] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:21.563178 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-f69baf79-3e0d-42c6-a35c-f8fd7d314707 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:21.574809 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69baf79-3e0d-42c6-a35c-f8fd7d314707 service cinder] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:21.640871 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e434163-6d40-499e-a19a-4577acdb3d79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:21.641690 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e434163-6d40-499e-a19a-4577acdb3d79 service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:21.642053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e434163-6d40-499e-a19a-4577acdb3d79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:21.642433 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e434163-6d40-499e-a19a-4577acdb3d79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:21.654203 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e434163-6d40-499e-a19a-4577acdb3d79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:21.656023 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e434163-6d40-499e-a19a-4577acdb3d79 service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:21.666565 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1317/2629] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:34:21 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:34:21.786539 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79f660-29f8-4c35-8f0d-04ddf41344c5 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:21.797785 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79f660-29f8-4c35-8f0d-04ddf41344c5 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:21.801436 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b311e3ce-44ae-474b-991e-cbef60abb0ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:21.802079 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311e3ce-44ae-474b-991e-cbef60abb0ae service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:21.802358 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311e3ce-44ae-474b-991e-cbef60abb0ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:21.802626 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311e3ce-44ae-474b-991e-cbef60abb0ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:21.864859 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b311e3ce-44ae-474b-991e-cbef60abb0ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:21.866312 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b311e3ce-44ae-474b-991e-cbef60abb0ae service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:21.875968 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1313/2630] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:34:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:34:22.979343 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e434163-6d40-499e-a19a-4577acdb3d79 service cinder] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:22.988402 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e434163-6d40-499e-a19a-4577acdb3d79 service cinder] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:22.992221 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a8831-2530-422f-8296-bdfb504900c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:22.992812 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a8831-2530-422f-8296-bdfb504900c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:22.993086 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a8831-2530-422f-8296-bdfb504900c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:22.993359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a8831-2530-422f-8296-bdfb504900c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:23.068464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16a8831-2530-422f-8296-bdfb504900c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:23.070027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16a8831-2530-422f-8296-bdfb504900c7 service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:23.080097 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1318/2631] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:34:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:34:23.698576 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b311e3ce-44ae-474b-991e-cbef60abb0ae service cinder] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:23.707572 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b311e3ce-44ae-474b-991e-cbef60abb0ae service cinder] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:23.711482 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb46b81-5530-4f63-8a8f-e92b2e804b67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:23.712097 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb46b81-5530-4f63-8a8f-e92b2e804b67 service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:23.712369 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb46b81-5530-4f63-8a8f-e92b2e804b67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:23.712633 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb46b81-5530-4f63-8a8f-e92b2e804b67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:23.776837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb46b81-5530-4f63-8a8f-e92b2e804b67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:23.778285 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb46b81-5530-4f63-8a8f-e92b2e804b67 service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:23.788254 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1314/2632] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:34:23 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:34:30.015245 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a8831-2530-422f-8296-bdfb504900c7 service cinder] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:30.096187 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a8831-2530-422f-8296-bdfb504900c7 service cinder] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:30.101506 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe480269-8d5b-4362-9dd9-738cc652e4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:30.102131 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe480269-8d5b-4362-9dd9-738cc652e4e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:30.102404 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe480269-8d5b-4362-9dd9-738cc652e4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:30.102673 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe480269-8d5b-4362-9dd9-738cc652e4e5 admin admin] PATH_INFO: `/v3/users/95fb255f4c4342daa13314fe74172887` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:30.103785 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe480269-8d5b-4362-9dd9-738cc652e4e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95fb255f4c4342daa13314fe74172887)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:30.105770 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe480269-8d5b-4362-9dd9-738cc652e4e5 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:30.150366 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:30.157276 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-fe480269-8d5b-4362-9dd9-738cc652e4e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95fb255f4c4342daa13314fe74172887'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:30.158202 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-fe480269-8d5b-4362-9dd9-738cc652e4e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95fb255f4c4342daa13314fe74172887'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:30.167419 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-fe480269-8d5b-4362-9dd9-738cc652e4e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95fb255f4c4342daa13314fe74172887'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:30.181674 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-fe480269-8d5b-4362-9dd9-738cc652e4e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95fb255f4c4342daa13314fe74172887'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:30.195230 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-fe480269-8d5b-4362-9dd9-738cc652e4e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95fb255f4c4342daa13314fe74172887'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:30.201903 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1319/2633] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:34:30 2022] DELETE /identity/v3/users/95fb255f4c4342daa13314fe74172887 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:30.206624 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec82b6-eaeb-4448-9d97-37a9233a8e6a None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:30.207119 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec82b6-eaeb-4448-9d97-37a9233a8e6a None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:30.207307 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec82b6-eaeb-4448-9d97-37a9233a8e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:30.228134 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-5eec82b6-eaeb-4448-9d97-37a9233a8e6a None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:34:30.229283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-5eec82b6-eaeb-4448-9d97-37a9233a8e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:30.293070 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1315/2634] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:34:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:34:30.305608 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe480269-8d5b-4362-9dd9-738cc652e4e5 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:30.309545 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-fe480269-8d5b-4362-9dd9-738cc652e4e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:30.374592 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe480269-8d5b-4362-9dd9-738cc652e4e5 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:30.380796 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8af6e-c3c6-4718-8b5f-b10cd5946e26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:30.381668 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8af6e-c3c6-4718-8b5f-b10cd5946e26 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:30.382054 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8af6e-c3c6-4718-8b5f-b10cd5946e26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:30.382414 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8af6e-c3c6-4718-8b5f-b10cd5946e26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:30.394794 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb8af6e-c3c6-4718-8b5f-b10cd5946e26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:30.396274 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb8af6e-c3c6-4718-8b5f-b10cd5946e26 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:30.406112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1320/2635] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:34:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:34:30.705028 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec82b6-eaeb-4448-9d97-37a9233a8e6a None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:30.706539 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-5eec82b6-eaeb-4448-9d97-37a9233a8e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:30.719115 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec82b6-eaeb-4448-9d97-37a9233a8e6a None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:30.724302 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b8836-de30-4698-8a70-9aec33944929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:30.724951 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b8836-de30-4698-8a70-9aec33944929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:30.725270 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b8836-de30-4698-8a70-9aec33944929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:30.725898 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b8836-de30-4698-8a70-9aec33944929 admin admin] PATH_INFO: `/v3/projects/7d935a9f23f64970ab825f32a069fdd3` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:30.727578 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b8836-de30-4698-8a70-9aec33944929 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d935a9f23f64970ab825f32a069fdd3)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:30.729837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b8836-de30-4698-8a70-9aec33944929 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:30.742624 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:30.779639 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-fd9b8836-de30-4698-8a70-9aec33944929 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d935a9f23f64970ab825f32a069fdd3', 'request_id': 'req-fd9b8836-de30-4698-8a70-9aec33944929'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:30.785988 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-fd9b8836-de30-4698-8a70-9aec33944929 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d935a9f23f64970ab825f32a069fdd3', 'request_id': 'req-fd9b8836-de30-4698-8a70-9aec33944929'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:30.795176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-fd9b8836-de30-4698-8a70-9aec33944929 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d935a9f23f64970ab825f32a069fdd3', 'request_id': 'req-fd9b8836-de30-4698-8a70-9aec33944929'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:30.805647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-fd9b8836-de30-4698-8a70-9aec33944929 admin admin] The token cache is being invalidate because project 7d935a9f23f64970ab825f32a069fdd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:34:30.805971 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-fd9b8836-de30-4698-8a70-9aec33944929 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:30.806589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-fd9b8836-de30-4698-8a70-9aec33944929 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:30.808109 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1316/2636] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:34:30 2022] DELETE /identity/v3/projects/7d935a9f23f64970ab825f32a069fdd3 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:30.812391 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8af6e-c3c6-4718-8b5f-b10cd5946e26 service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:30.822475 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8af6e-c3c6-4718-8b5f-b10cd5946e26 service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:30.882645 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e4f02-7eaa-4c1d-a63c-e9f679139f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:30.883265 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e4f02-7eaa-4c1d-a63c-e9f679139f33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:30.883526 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e4f02-7eaa-4c1d-a63c-e9f679139f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:30.883809 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e4f02-7eaa-4c1d-a63c-e9f679139f33 admin admin] PATH_INFO: `/v3/users/2ddc39d6eb0c470c9184f16b76f1f3fe` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:30.884965 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e4f02-7eaa-4c1d-a63c-e9f679139f33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ddc39d6eb0c470c9184f16b76f1f3fe)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:30.886925 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e4f02-7eaa-4c1d-a63c-e9f679139f33 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:30.935630 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-fe1e4f02-7eaa-4c1d-a63c-e9f679139f33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ddc39d6eb0c470c9184f16b76f1f3fe'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:30.936366 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-fe1e4f02-7eaa-4c1d-a63c-e9f679139f33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ddc39d6eb0c470c9184f16b76f1f3fe'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:30.945105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-fe1e4f02-7eaa-4c1d-a63c-e9f679139f33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ddc39d6eb0c470c9184f16b76f1f3fe'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:30.950357 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:30.958841 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-fe1e4f02-7eaa-4c1d-a63c-e9f679139f33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ddc39d6eb0c470c9184f16b76f1f3fe'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:30.966131 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-fe1e4f02-7eaa-4c1d-a63c-e9f679139f33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ddc39d6eb0c470c9184f16b76f1f3fe'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:30.974081 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1321/2637] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:34:30 2022] DELETE /identity/v3/users/2ddc39d6eb0c470c9184f16b76f1f3fe => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:31.288240 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b8836-de30-4698-8a70-9aec33944929 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:31.289863 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-fd9b8836-de30-4698-8a70-9aec33944929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:31.311420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b8836-de30-4698-8a70-9aec33944929 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:31.385417 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91db745-3b26-4ae5-b77a-94aa408b4bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:31.386056 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91db745-3b26-4ae5-b77a-94aa408b4bce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:31.386329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91db745-3b26-4ae5-b77a-94aa408b4bce admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:31.386602 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91db745-3b26-4ae5-b77a-94aa408b4bce admin admin] PATH_INFO: `/v3/projects/9b661e4a3f9a4c32ad1cb83cae23d40b` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:31.387771 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91db745-3b26-4ae5-b77a-94aa408b4bce admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b661e4a3f9a4c32ad1cb83cae23d40b)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:31.390689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91db745-3b26-4ae5-b77a-94aa408b4bce admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:31.447316 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-e91db745-3b26-4ae5-b77a-94aa408b4bce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b661e4a3f9a4c32ad1cb83cae23d40b', 'request_id': 'req-e91db745-3b26-4ae5-b77a-94aa408b4bce'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:31.457928 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-e91db745-3b26-4ae5-b77a-94aa408b4bce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b661e4a3f9a4c32ad1cb83cae23d40b', 'request_id': 'req-e91db745-3b26-4ae5-b77a-94aa408b4bce'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:31.469183 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-e91db745-3b26-4ae5-b77a-94aa408b4bce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b661e4a3f9a4c32ad1cb83cae23d40b', 'request_id': 'req-e91db745-3b26-4ae5-b77a-94aa408b4bce'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:31.480919 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-e91db745-3b26-4ae5-b77a-94aa408b4bce admin admin] The token cache is being invalidate because project 9b661e4a3f9a4c32ad1cb83cae23d40b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:34:31.481315 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-e91db745-3b26-4ae5-b77a-94aa408b4bce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:31.481983 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-e91db745-3b26-4ae5-b77a-94aa408b4bce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:31.483612 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1317/2638] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:34:31 2022] DELETE /identity/v3/projects/9b661e4a3f9a4c32ad1cb83cae23d40b => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:31.487898 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e4f02-7eaa-4c1d-a63c-e9f679139f33 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:31.489907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-fe1e4f02-7eaa-4c1d-a63c-e9f679139f33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:31.504230 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e4f02-7eaa-4c1d-a63c-e9f679139f33 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:31.569880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d1814-b380-40da-857f-3184312e88f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:31.570406 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d1814-b380-40da-857f-3184312e88f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:31.570720 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d1814-b380-40da-857f-3184312e88f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:31.570993 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d1814-b380-40da-857f-3184312e88f3 admin admin] PATH_INFO: `/v3/users/d9687acf6a55401a95677fb4b86bcb1b` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:31.572291 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d1814-b380-40da-857f-3184312e88f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9687acf6a55401a95677fb4b86bcb1b)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:31.574245 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d1814-b380-40da-857f-3184312e88f3 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:31.626143 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-869d1814-b380-40da-857f-3184312e88f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9687acf6a55401a95677fb4b86bcb1b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:31.626875 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-869d1814-b380-40da-857f-3184312e88f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9687acf6a55401a95677fb4b86bcb1b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:31.638458 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-869d1814-b380-40da-857f-3184312e88f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9687acf6a55401a95677fb4b86bcb1b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:31.651632 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-869d1814-b380-40da-857f-3184312e88f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9687acf6a55401a95677fb4b86bcb1b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:31.658922 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-869d1814-b380-40da-857f-3184312e88f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9687acf6a55401a95677fb4b86bcb1b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:31.664909 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1322/2639] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:34:31 2022] DELETE /identity/v3/users/d9687acf6a55401a95677fb4b86bcb1b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:31.995198 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91db745-3b26-4ae5-b77a-94aa408b4bce admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:31.997011 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-e91db745-3b26-4ae5-b77a-94aa408b4bce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:32.011163 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91db745-3b26-4ae5-b77a-94aa408b4bce admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:32.049079 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:32.088138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d7fef-d9b3-44a5-80d2-20b2ef382b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:32.088816 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d7fef-d9b3-44a5-80d2-20b2ef382b60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:32.089093 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d7fef-d9b3-44a5-80d2-20b2ef382b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:32.089366 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d7fef-d9b3-44a5-80d2-20b2ef382b60 admin admin] PATH_INFO: `/v3/projects/e16ef088d0934313a3d704d4ad2276e3` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:32.090711 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098d7fef-d9b3-44a5-80d2-20b2ef382b60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e16ef088d0934313a3d704d4ad2276e3)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:32.093117 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098d7fef-d9b3-44a5-80d2-20b2ef382b60 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:32.162631 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-098d7fef-d9b3-44a5-80d2-20b2ef382b60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e16ef088d0934313a3d704d4ad2276e3', 'request_id': 'req-098d7fef-d9b3-44a5-80d2-20b2ef382b60'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:32.169156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-098d7fef-d9b3-44a5-80d2-20b2ef382b60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e16ef088d0934313a3d704d4ad2276e3', 'request_id': 'req-098d7fef-d9b3-44a5-80d2-20b2ef382b60'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:32.177690 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-098d7fef-d9b3-44a5-80d2-20b2ef382b60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e16ef088d0934313a3d704d4ad2276e3', 'request_id': 'req-098d7fef-d9b3-44a5-80d2-20b2ef382b60'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:32.199201 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-098d7fef-d9b3-44a5-80d2-20b2ef382b60 admin admin] The token cache is being invalidate because project e16ef088d0934313a3d704d4ad2276e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:34:32.199562 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-098d7fef-d9b3-44a5-80d2-20b2ef382b60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:32.200252 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-098d7fef-d9b3-44a5-80d2-20b2ef382b60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:34:32.202063 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1318/2640] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:34:31 2022] DELETE /identity/v3/projects/e16ef088d0934313a3d704d4ad2276e3 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:32.207471 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6dbe3-e349-4ee3-829a-7ff37c313d2f None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:32.207748 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6dbe3-e349-4ee3-829a-7ff37c313d2f None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:32.208077 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6dbe3-e349-4ee3-829a-7ff37c313d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:32.228648 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-1cd6dbe3-e349-4ee3-829a-7ff37c313d2f None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:34:32.229861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-1cd6dbe3-e349-4ee3-829a-7ff37c313d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:32.280581 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:32.293671 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1323/2641] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:34:32.302501 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df834e-03af-4ca1-8a8f-ad2038019389 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:32.303037 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df834e-03af-4ca1-8a8f-ad2038019389 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:32.303037 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df834e-03af-4ca1-8a8f-ad2038019389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:32.328868 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-f9df834e-03af-4ca1-8a8f-ad2038019389 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:34:32.330146 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-f9df834e-03af-4ca1-8a8f-ad2038019389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:32.348832 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1319/2642] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:34:32.358548 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6dbe3-e349-4ee3-829a-7ff37c313d2f None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:32.369004 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6dbe3-e349-4ee3-829a-7ff37c313d2f None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:32.374266 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e01dc-fefb-42d5-b4f7-ae117ca60f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:32.374905 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e01dc-fefb-42d5-b4f7-ae117ca60f61 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:32.375183 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e01dc-fefb-42d5-b4f7-ae117ca60f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:32.375479 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e01dc-fefb-42d5-b4f7-ae117ca60f61 admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:32.376170 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e01dc-fefb-42d5-b4f7-ae117ca60f61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:34:32.376472 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e01dc-fefb-42d5-b4f7-ae117ca60f61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:32.378091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e01dc-fefb-42d5-b4f7-ae117ca60f61 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:32.392092 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1324/2643] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:34:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:34:32.397068 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df834e-03af-4ca1-8a8f-ad2038019389 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:32.414729 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df834e-03af-4ca1-8a8f-ad2038019389 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:32.420169 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d97ae0-71b6-47db-be9b-1fccd1c016ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:32.420910 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d97ae0-71b6-47db-be9b-1fccd1c016ff admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:32.421246 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d97ae0-71b6-47db-be9b-1fccd1c016ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:32.421533 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d97ae0-71b6-47db-be9b-1fccd1c016ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:32.422323 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d97ae0-71b6-47db-be9b-1fccd1c016ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:32.424969 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d97ae0-71b6-47db-be9b-1fccd1c016ff admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:32.461881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1320/2644] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:34:32 2022] POST /identity/v3/projects => generated 367 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:34:32.466529 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e01dc-fefb-42d5-b4f7-ae117ca60f61 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:32.477666 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e01dc-fefb-42d5-b4f7-ae117ca60f61 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:32.535544 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e8182-e791-4f02-9d91-cb376026cf94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:32.536344 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e8182-e791-4f02-9d91-cb376026cf94 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:32.536466 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e8182-e791-4f02-9d91-cb376026cf94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:32.536695 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e8182-e791-4f02-9d91-cb376026cf94 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:32.537311 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e8182-e791-4f02-9d91-cb376026cf94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:32.539208 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e8182-e791-4f02-9d91-cb376026cf94 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:32.562546 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1325/2645] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:34:32 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:34:32.567065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d97ae0-71b6-47db-be9b-1fccd1c016ff admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:32.577520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d97ae0-71b6-47db-be9b-1fccd1c016ff admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:32.582136 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e9918-bc6a-4689-b15c-0e0bd4c6a0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:32.582728 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e9918-bc6a-4689-b15c-0e0bd4c6a0ee admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:32.583062 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e9918-bc6a-4689-b15c-0e0bd4c6a0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:32.583332 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e9918-bc6a-4689-b15c-0e0bd4c6a0ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:32.583969 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e9918-bc6a-4689-b15c-0e0bd4c6a0ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:32.585747 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e9918-bc6a-4689-b15c-0e0bd4c6a0ee admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:32.595539 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1321/2646] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:34:32.599471 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e8182-e791-4f02-9d91-cb376026cf94 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:32.610177 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e8182-e791-4f02-9d91-cb376026cf94 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:32.615618 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec04fe4-d32b-4f0c-9822-d8cb88245c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:32.616256 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec04fe4-d32b-4f0c-9822-d8cb88245c1c admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:32.616530 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec04fe4-d32b-4f0c-9822-d8cb88245c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:32.616792 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec04fe4-d32b-4f0c-9822-d8cb88245c1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:32.617427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec04fe4-d32b-4f0c-9822-d8cb88245c1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:32.619018 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec04fe4-d32b-4f0c-9822-d8cb88245c1c admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:32.635101 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1326/2647] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:34:32.639258 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e9918-bc6a-4689-b15c-0e0bd4c6a0ee admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:32.649306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e9918-bc6a-4689-b15c-0e0bd4c6a0ee admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:32.654192 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebef691-982c-432b-82f4-1007a7efc201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:32.654832 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebef691-982c-432b-82f4-1007a7efc201 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:32.655101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebef691-982c-432b-82f4-1007a7efc201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:32.655375 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebef691-982c-432b-82f4-1007a7efc201 admin admin] PATH_INFO: `/v3/projects/5dfe5e252c81484d93623b605e48746e/users/a866b78df6b542c08bdec06232c6edd0/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:32.671342 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebef691-982c-432b-82f4-1007a7efc201 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dfe5e252c81484d93623b605e48746e, user_id=a866b78df6b542c08bdec06232c6edd0, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:32.674002 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebef691-982c-432b-82f4-1007a7efc201 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:32.689918 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1322/2648] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:34:32 2022] PUT /identity/v3/projects/5dfe5e252c81484d93623b605e48746e/users/a866b78df6b542c08bdec06232c6edd0/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:34:32.709059 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23bd16-438d-43b5-8ee4-38335a5933c2 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:32.709350 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23bd16-438d-43b5-8ee4-38335a5933c2 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:32.709644 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23bd16-438d-43b5-8ee4-38335a5933c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:32.741292 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-6a23bd16-438d-43b5-8ee4-38335a5933c2 None None] MFA Rules not processed for user `a866b78df6b542c08bdec06232c6edd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:34:32.742550 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-6a23bd16-438d-43b5-8ee4-38335a5933c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:32.810514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1327/2649] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:34:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:34:32.822193 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebef691-982c-432b-82f4-1007a7efc201 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:34:32.824147 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-2ebef691-982c-432b-82f4-1007a7efc201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:34:32.883480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebef691-982c-432b-82f4-1007a7efc201 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:34:32.887232 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a2e37-393c-4d87-ae55-c603746d5572 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f1bf93cff6c46e0801bb827203cb5ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:34:32.887862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a2e37-393c-4d87-ae55-c603746d5572 service glance] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:34:32.888145 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a2e37-393c-4d87-ae55-c603746d5572 service glance] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:34:32.888607 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a2e37-393c-4d87-ae55-c603746d5572 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:34:32.896841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a2e37-393c-4d87-ae55-c603746d5572 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:34:32.898270 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a2e37-393c-4d87-ae55-c603746d5572 service glance] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:34:32.902968 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:34:32.908179 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1323/2650] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:34:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:34:42.807405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=73,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:09.299150 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23bd16-438d-43b5-8ee4-38335a5933c2 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:09.301333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-6a23bd16-438d-43b5-8ee4-38335a5933c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:09.371758 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23bd16-438d-43b5-8ee4-38335a5933c2 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:09.376975 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabbf17-a4fa-433f-9e7d-bd95c2b62fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:09.377661 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabbf17-a4fa-433f-9e7d-bd95c2b62fba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:09.377937 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabbf17-a4fa-433f-9e7d-bd95c2b62fba admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:09.378204 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabbf17-a4fa-433f-9e7d-bd95c2b62fba admin admin] PATH_INFO: `/v3/users/760985d5610c4b61a61cac2dbf9449d2` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:09.379414 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dabbf17-a4fa-433f-9e7d-bd95c2b62fba admin admin] RBAC: Authorizing `identity:delete_user(user_id=760985d5610c4b61a61cac2dbf9449d2)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:09.381360 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dabbf17-a4fa-433f-9e7d-bd95c2b62fba admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:09.436017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8dabbf17-a4fa-433f-9e7d-bd95c2b62fba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '760985d5610c4b61a61cac2dbf9449d2'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:09.436672 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8dabbf17-a4fa-433f-9e7d-bd95c2b62fba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '760985d5610c4b61a61cac2dbf9449d2'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:09.445924 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8dabbf17-a4fa-433f-9e7d-bd95c2b62fba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '760985d5610c4b61a61cac2dbf9449d2'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:09.466144 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8dabbf17-a4fa-433f-9e7d-bd95c2b62fba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '760985d5610c4b61a61cac2dbf9449d2'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:09.473531 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-8dabbf17-a4fa-433f-9e7d-bd95c2b62fba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '760985d5610c4b61a61cac2dbf9449d2'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:09.480535 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1328/2651] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:35:09 2022] DELETE /identity/v3/users/760985d5610c4b61a61cac2dbf9449d2 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:35:09.485426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120b13e-1e7f-4f71-8ecc-d2405b5bfe6e None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:09.485894 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120b13e-1e7f-4f71-8ecc-d2405b5bfe6e None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:09.486078 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120b13e-1e7f-4f71-8ecc-d2405b5bfe6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:09.505625 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-a120b13e-1e7f-4f71-8ecc-d2405b5bfe6e None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:09.508554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-a120b13e-1e7f-4f71-8ecc-d2405b5bfe6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:09.579743 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1324/2652] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:35:09.593176 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabbf17-a4fa-433f-9e7d-bd95c2b62fba admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:09.594761 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-8dabbf17-a4fa-433f-9e7d-bd95c2b62fba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:09.667539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabbf17-a4fa-433f-9e7d-bd95c2b62fba admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:09.672524 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae1d45-2ece-451a-8f7a-a692b71b84ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:09.673418 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae1d45-2ece-451a-8f7a-a692b71b84ec service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:09.673720 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae1d45-2ece-451a-8f7a-a692b71b84ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:09.674160 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae1d45-2ece-451a-8f7a-a692b71b84ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:09.690188 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae1d45-2ece-451a-8f7a-a692b71b84ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:09.692743 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae1d45-2ece-451a-8f7a-a692b71b84ec service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:09.708879 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1329/2653] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:35:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:35:09.741241 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120b13e-1e7f-4f71-8ecc-d2405b5bfe6e None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:09.742689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-a120b13e-1e7f-4f71-8ecc-d2405b5bfe6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:09.755013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120b13e-1e7f-4f71-8ecc-d2405b5bfe6e None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:09.759631 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecaba4c-4b99-48ad-869d-de748b9fe0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:09.760314 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecaba4c-4b99-48ad-869d-de748b9fe0d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:09.760592 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecaba4c-4b99-48ad-869d-de748b9fe0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:09.760862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecaba4c-4b99-48ad-869d-de748b9fe0d6 admin admin] PATH_INFO: `/v3/users/c35cfc1e230e4b4593a63e9475fb6747` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:09.762083 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecaba4c-4b99-48ad-869d-de748b9fe0d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c35cfc1e230e4b4593a63e9475fb6747)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:09.763961 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecaba4c-4b99-48ad-869d-de748b9fe0d6 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:09.816523 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-eecaba4c-4b99-48ad-869d-de748b9fe0d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c35cfc1e230e4b4593a63e9475fb6747'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:09.825611 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-eecaba4c-4b99-48ad-869d-de748b9fe0d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c35cfc1e230e4b4593a63e9475fb6747'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:09.840761 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-eecaba4c-4b99-48ad-869d-de748b9fe0d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c35cfc1e230e4b4593a63e9475fb6747'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:09.841673 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-eecaba4c-4b99-48ad-869d-de748b9fe0d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c35cfc1e230e4b4593a63e9475fb6747'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:09.849921 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-eecaba4c-4b99-48ad-869d-de748b9fe0d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c35cfc1e230e4b4593a63e9475fb6747'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:09.857517 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1325/2654] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:35:09 2022] DELETE /identity/v3/users/c35cfc1e230e4b4593a63e9475fb6747 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:35:09.863517 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e8d96-edb7-4ccb-9186-86a3c53e9544 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:09.864125 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e8d96-edb7-4ccb-9186-86a3c53e9544 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:09.864510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e8d96-edb7-4ccb-9186-86a3c53e9544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:09.892104 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-d95e8d96-edb7-4ccb-9186-86a3c53e9544 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:09.893582 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-d95e8d96-edb7-4ccb-9186-86a3c53e9544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:09.912680 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:09.958040 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1330/2655] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:35:09.968321 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecaba4c-4b99-48ad-869d-de748b9fe0d6 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:09.969754 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-eecaba4c-4b99-48ad-869d-de748b9fe0d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:10.005754 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e8d96-edb7-4ccb-9186-86a3c53e9544 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:10.007605 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-d95e8d96-edb7-4ccb-9186-86a3c53e9544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:10.022982 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e8d96-edb7-4ccb-9186-86a3c53e9544 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:10.029544 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31180b-78cc-4cd2-abad-464b10502eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:10.030198 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31180b-78cc-4cd2-abad-464b10502eda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:10.030513 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31180b-78cc-4cd2-abad-464b10502eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:10.030740 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31180b-78cc-4cd2-abad-464b10502eda admin admin] PATH_INFO: `/v3/projects/f8c314d0382943438b2182f2800101e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:10.031880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca31180b-78cc-4cd2-abad-464b10502eda admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8c314d0382943438b2182f2800101e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:10.034681 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca31180b-78cc-4cd2-abad-464b10502eda admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:10.043331 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecaba4c-4b99-48ad-869d-de748b9fe0d6 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:10.047622 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e3598-82f1-4734-bce3-e7c05862bb0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:10.048357 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e3598-82f1-4734-bce3-e7c05862bb0b service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:10.048911 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e3598-82f1-4734-bce3-e7c05862bb0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:10.048911 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e3598-82f1-4734-bce3-e7c05862bb0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:10.060107 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e3598-82f1-4734-bce3-e7c05862bb0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:10.061570 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e3598-82f1-4734-bce3-e7c05862bb0b service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:10.073249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1326/2656] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:35:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:35:10.092267 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ca31180b-78cc-4cd2-abad-464b10502eda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8c314d0382943438b2182f2800101e1', 'request_id': 'req-ca31180b-78cc-4cd2-abad-464b10502eda'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:10.098793 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ca31180b-78cc-4cd2-abad-464b10502eda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8c314d0382943438b2182f2800101e1', 'request_id': 'req-ca31180b-78cc-4cd2-abad-464b10502eda'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:10.108297 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ca31180b-78cc-4cd2-abad-464b10502eda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8c314d0382943438b2182f2800101e1', 'request_id': 'req-ca31180b-78cc-4cd2-abad-464b10502eda'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:10.119210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ca31180b-78cc-4cd2-abad-464b10502eda admin admin] The token cache is being invalidate because project f8c314d0382943438b2182f2800101e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:35:10.119503 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ca31180b-78cc-4cd2-abad-464b10502eda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:10.120270 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ca31180b-78cc-4cd2-abad-464b10502eda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:10.122288 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1331/2657] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:35:10 2022] DELETE /identity/v3/projects/f8c314d0382943438b2182f2800101e1 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:35:10.376662 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e3598-82f1-4734-bce3-e7c05862bb0b service neutron] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:10.378704 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-c53e3598-82f1-4734-bce3-e7c05862bb0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:10.393463 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e3598-82f1-4734-bce3-e7c05862bb0b service neutron] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:10.406640 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:10.456506 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c41698-b60d-4b73-8416-0e661c43c27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:10.457172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c41698-b60d-4b73-8416-0e661c43c27d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:10.457451 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c41698-b60d-4b73-8416-0e661c43c27d admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:10.457702 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c41698-b60d-4b73-8416-0e661c43c27d admin admin] PATH_INFO: `/v3/projects/d190cd833ca743c08e903166138f008f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:10.458958 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c41698-b60d-4b73-8416-0e661c43c27d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d190cd833ca743c08e903166138f008f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:10.461033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c41698-b60d-4b73-8416-0e661c43c27d admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:10.514620 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-56c41698-b60d-4b73-8416-0e661c43c27d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd190cd833ca743c08e903166138f008f', 'request_id': 'req-56c41698-b60d-4b73-8416-0e661c43c27d'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:10.521494 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-56c41698-b60d-4b73-8416-0e661c43c27d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd190cd833ca743c08e903166138f008f', 'request_id': 'req-56c41698-b60d-4b73-8416-0e661c43c27d'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:10.531575 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-56c41698-b60d-4b73-8416-0e661c43c27d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd190cd833ca743c08e903166138f008f', 'request_id': 'req-56c41698-b60d-4b73-8416-0e661c43c27d'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:10.544460 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-56c41698-b60d-4b73-8416-0e661c43c27d admin admin] The token cache is being invalidate because project d190cd833ca743c08e903166138f008f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:35:10.544764 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-56c41698-b60d-4b73-8416-0e661c43c27d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:10.545520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-56c41698-b60d-4b73-8416-0e661c43c27d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:10.547931 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1327/2658] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:35:10 2022] DELETE /identity/v3/projects/d190cd833ca743c08e903166138f008f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:35:10.551240 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31180b-78cc-4cd2-abad-464b10502eda admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:10.553600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-ca31180b-78cc-4cd2-abad-464b10502eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:10.567387 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31180b-78cc-4cd2-abad-464b10502eda admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:10.635603 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e2977-e095-4362-8f0a-cc2f38d85185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:10.636191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e2977-e095-4362-8f0a-cc2f38d85185 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:10.636455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e2977-e095-4362-8f0a-cc2f38d85185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:10.636724 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e2977-e095-4362-8f0a-cc2f38d85185 admin admin] PATH_INFO: `/v3/users/69046982b02b44eeba95903048ea73eb` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:10.637843 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539e2977-e095-4362-8f0a-cc2f38d85185 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69046982b02b44eeba95903048ea73eb)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:10.639774 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539e2977-e095-4362-8f0a-cc2f38d85185 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:10.688644 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-539e2977-e095-4362-8f0a-cc2f38d85185 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69046982b02b44eeba95903048ea73eb'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:10.689287 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-539e2977-e095-4362-8f0a-cc2f38d85185 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69046982b02b44eeba95903048ea73eb'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:10.691016 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:10.698360 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-539e2977-e095-4362-8f0a-cc2f38d85185 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69046982b02b44eeba95903048ea73eb'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:10.713223 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-539e2977-e095-4362-8f0a-cc2f38d85185 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69046982b02b44eeba95903048ea73eb'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:10.720595 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-539e2977-e095-4362-8f0a-cc2f38d85185 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69046982b02b44eeba95903048ea73eb'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:10.728611 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1332/2659] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:35:10 2022] DELETE /identity/v3/users/69046982b02b44eeba95903048ea73eb => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:35:11.020030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c41698-b60d-4b73-8416-0e661c43c27d admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:11.021955 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-56c41698-b60d-4b73-8416-0e661c43c27d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:11.040617 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c41698-b60d-4b73-8416-0e661c43c27d admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:11.102759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b7175-cf88-4be7-88a2-2b1c0124a954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:11.103422 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b7175-cf88-4be7-88a2-2b1c0124a954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:11.103700 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b7175-cf88-4be7-88a2-2b1c0124a954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:11.104014 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b7175-cf88-4be7-88a2-2b1c0124a954 admin admin] PATH_INFO: `/v3/projects/f1929dde909144c38aa344ba9efb316f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:11.105187 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b7175-cf88-4be7-88a2-2b1c0124a954 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1929dde909144c38aa344ba9efb316f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:11.107196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b7175-cf88-4be7-88a2-2b1c0124a954 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:11.161057 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:11.162569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-657b7175-cf88-4be7-88a2-2b1c0124a954 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1929dde909144c38aa344ba9efb316f', 'request_id': 'req-657b7175-cf88-4be7-88a2-2b1c0124a954'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:11.170424 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-657b7175-cf88-4be7-88a2-2b1c0124a954 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1929dde909144c38aa344ba9efb316f', 'request_id': 'req-657b7175-cf88-4be7-88a2-2b1c0124a954'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:11.180843 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-657b7175-cf88-4be7-88a2-2b1c0124a954 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1929dde909144c38aa344ba9efb316f', 'request_id': 'req-657b7175-cf88-4be7-88a2-2b1c0124a954'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:11.191974 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-657b7175-cf88-4be7-88a2-2b1c0124a954 admin admin] The token cache is being invalidate because project f1929dde909144c38aa344ba9efb316f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:35:11.192270 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-657b7175-cf88-4be7-88a2-2b1c0124a954 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:11.192916 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-657b7175-cf88-4be7-88a2-2b1c0124a954 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:11.194452 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1328/2660] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:35:11 2022] DELETE /identity/v3/projects/f1929dde909144c38aa344ba9efb316f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:35:11.200370 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d391d5c-e679-4051-bfb4-1a41f2f3a36d None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:11.200894 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d391d5c-e679-4051-bfb4-1a41f2f3a36d None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:11.201225 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d391d5c-e679-4051-bfb4-1a41f2f3a36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:11.222132 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-2d391d5c-e679-4051-bfb4-1a41f2f3a36d None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:11.223339 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-2d391d5c-e679-4051-bfb4-1a41f2f3a36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:11.285882 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1333/2661] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:35:11.294109 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f6190-beb7-4277-9670-48077e30b112 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:11.294393 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f6190-beb7-4277-9670-48077e30b112 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:11.294665 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f6190-beb7-4277-9670-48077e30b112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:11.315650 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-ff9f6190-beb7-4277-9670-48077e30b112 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:11.316901 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-ff9f6190-beb7-4277-9670-48077e30b112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:11.326081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1329/2662] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:35:11.331164 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d391d5c-e679-4051-bfb4-1a41f2f3a36d None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:11.342643 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d391d5c-e679-4051-bfb4-1a41f2f3a36d None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:11.348468 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12784208-8750-4aed-9a33-02a2f62555df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:11.349072 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12784208-8750-4aed-9a33-02a2f62555df admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:11.349356 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12784208-8750-4aed-9a33-02a2f62555df admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:11.349661 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12784208-8750-4aed-9a33-02a2f62555df admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:11.350294 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12784208-8750-4aed-9a33-02a2f62555df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:35:11.350610 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12784208-8750-4aed-9a33-02a2f62555df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:11.353120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12784208-8750-4aed-9a33-02a2f62555df admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:11.365053 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1334/2663] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:35:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:35:11.370309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f6190-beb7-4277-9670-48077e30b112 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:11.379933 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f6190-beb7-4277-9670-48077e30b112 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:11.385129 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee04f00-daf7-4071-9f66-e689e8996048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:11.385720 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee04f00-daf7-4071-9f66-e689e8996048 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:11.385993 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee04f00-daf7-4071-9f66-e689e8996048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:11.386262 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee04f00-daf7-4071-9f66-e689e8996048 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:11.386874 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee04f00-daf7-4071-9f66-e689e8996048 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:11.388811 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee04f00-daf7-4071-9f66-e689e8996048 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:11.420010 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1330/2664] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:35:11 2022] POST /identity/v3/projects => generated 357 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:35:11.426537 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12784208-8750-4aed-9a33-02a2f62555df admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:11.438384 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12784208-8750-4aed-9a33-02a2f62555df admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:11.499522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d410d-a348-4346-a652-6966a4b3e1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:11.500153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d410d-a348-4346-a652-6966a4b3e1d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:11.500430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d410d-a348-4346-a652-6966a4b3e1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:11.500699 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d410d-a348-4346-a652-6966a4b3e1d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:11.501328 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525d410d-a348-4346-a652-6966a4b3e1d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:11.503247 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525d410d-a348-4346-a652-6966a4b3e1d2 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:11.525944 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1335/2665] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:35:11 2022] POST /identity/v3/users => generated 279 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:35:11.530307 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee04f00-daf7-4071-9f66-e689e8996048 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:11.541442 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee04f00-daf7-4071-9f66-e689e8996048 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:11.546840 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6126e9-6029-4e0a-bdb8-3545c651d57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:11.547479 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6126e9-6029-4e0a-bdb8-3545c651d57d admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:11.547750 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6126e9-6029-4e0a-bdb8-3545c651d57d admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:11.548061 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6126e9-6029-4e0a-bdb8-3545c651d57d admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:11.548710 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6126e9-6029-4e0a-bdb8-3545c651d57d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:11.550338 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6126e9-6029-4e0a-bdb8-3545c651d57d admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:11.561322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1331/2666] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:35:11.565489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d410d-a348-4346-a652-6966a4b3e1d2 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:11.576710 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d410d-a348-4346-a652-6966a4b3e1d2 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:11.582687 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354401a-d3ef-45fd-ae2c-7f20e505bde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:11.583291 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354401a-d3ef-45fd-ae2c-7f20e505bde8 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:11.583571 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354401a-d3ef-45fd-ae2c-7f20e505bde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:11.583844 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354401a-d3ef-45fd-ae2c-7f20e505bde8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:11.584519 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5354401a-d3ef-45fd-ae2c-7f20e505bde8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:11.586600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5354401a-d3ef-45fd-ae2c-7f20e505bde8 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:11.599990 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1336/2667] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:35:11.608151 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6126e9-6029-4e0a-bdb8-3545c651d57d admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:11.623668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6126e9-6029-4e0a-bdb8-3545c651d57d admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:11.632136 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c762a3-b427-4ec4-b1d2-f5518c501f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:11.633895 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c762a3-b427-4ec4-b1d2-f5518c501f9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:11.634175 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c762a3-b427-4ec4-b1d2-f5518c501f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:11.634447 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c762a3-b427-4ec4-b1d2-f5518c501f9d admin admin] PATH_INFO: `/v3/projects/4753c87e6a6b49cd9fde2a7350afab04/users/702a25fbad0e40a385de9f5c2821fe13/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:11.650764 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c762a3-b427-4ec4-b1d2-f5518c501f9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4753c87e6a6b49cd9fde2a7350afab04, user_id=702a25fbad0e40a385de9f5c2821fe13, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:11.653454 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c762a3-b427-4ec4-b1d2-f5518c501f9d admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:11.671772 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1332/2668] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:35:11 2022] PUT /identity/v3/projects/4753c87e6a6b49cd9fde2a7350afab04/users/702a25fbad0e40a385de9f5c2821fe13/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:35:11.684151 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74eff81-9ec6-4873-bfb6-f90bb1a17591 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:11.684606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74eff81-9ec6-4873-bfb6-f90bb1a17591 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:11.684999 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74eff81-9ec6-4873-bfb6-f90bb1a17591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:11.717029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-a74eff81-9ec6-4873-bfb6-f90bb1a17591 None None] MFA Rules not processed for user `702a25fbad0e40a385de9f5c2821fe13`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:11.718275 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a74eff81-9ec6-4873-bfb6-f90bb1a17591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:11.738138 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:11.788931 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1337/2669] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:11 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:35:11.794056 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c762a3-b427-4ec4-b1d2-f5518c501f9d admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:11.865194 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c762a3-b427-4ec4-b1d2-f5518c501f9d admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:11.871763 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f7074-6c21-4f0b-9ee4-dbfc08412455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:11.872438 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f7074-6c21-4f0b-9ee4-dbfc08412455 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:11.872702 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f7074-6c21-4f0b-9ee4-dbfc08412455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:11.872989 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f7074-6c21-4f0b-9ee4-dbfc08412455 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:11.873609 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f7074-6c21-4f0b-9ee4-dbfc08412455 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:11.875542 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f7074-6c21-4f0b-9ee4-dbfc08412455 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:11.910841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1333/2670] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:35:11 2022] POST /identity/v3/projects => generated 357 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:35:11.915642 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74eff81-9ec6-4873-bfb6-f90bb1a17591 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:11.927538 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74eff81-9ec6-4873-bfb6-f90bb1a17591 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:11.997679 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698c2cd-5db9-4a7d-8d8a-157c4d438823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:11.998352 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698c2cd-5db9-4a7d-8d8a-157c4d438823 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:11.998634 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698c2cd-5db9-4a7d-8d8a-157c4d438823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:11.998903 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698c2cd-5db9-4a7d-8d8a-157c4d438823 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:11.999559 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b698c2cd-5db9-4a7d-8d8a-157c4d438823 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:12.001505 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b698c2cd-5db9-4a7d-8d8a-157c4d438823 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:12.024668 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1338/2671] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:35:11 2022] POST /identity/v3/users => generated 285 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:35:12.028769 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f7074-6c21-4f0b-9ee4-dbfc08412455 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:12.039252 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f7074-6c21-4f0b-9ee4-dbfc08412455 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:12.043819 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0df07-d8a2-462f-8332-3b7a0265b5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:12.044454 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0df07-d8a2-462f-8332-3b7a0265b5e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:12.044734 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0df07-d8a2-462f-8332-3b7a0265b5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:12.045007 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0df07-d8a2-462f-8332-3b7a0265b5e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:12.045667 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a0df07-d8a2-462f-8332-3b7a0265b5e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:12.047272 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a0df07-d8a2-462f-8332-3b7a0265b5e3 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:12.058564 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1334/2672] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:35:12.063246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698c2cd-5db9-4a7d-8d8a-157c4d438823 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:12.079107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698c2cd-5db9-4a7d-8d8a-157c4d438823 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:12.086683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce97d588-07d9-462c-bdd5-e5fc7a4eae21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:12.087351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97d588-07d9-462c-bdd5-e5fc7a4eae21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:12.087644 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97d588-07d9-462c-bdd5-e5fc7a4eae21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:12.087947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97d588-07d9-462c-bdd5-e5fc7a4eae21 admin admin] PATH_INFO: `/v3/projects/c205060c98094a2c9969cc9ba712edb3/users/a2966ff64c454a82a88f5349de3b2f98/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:12.105378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce97d588-07d9-462c-bdd5-e5fc7a4eae21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c205060c98094a2c9969cc9ba712edb3, user_id=a2966ff64c454a82a88f5349de3b2f98, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:12.108188 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce97d588-07d9-462c-bdd5-e5fc7a4eae21 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:12.127282 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1339/2673] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:35:12 2022] PUT /identity/v3/projects/c205060c98094a2c9969cc9ba712edb3/users/a2966ff64c454a82a88f5349de3b2f98/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:35:12.131445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0df07-d8a2-462f-8332-3b7a0265b5e3 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:12.141887 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0df07-d8a2-462f-8332-3b7a0265b5e3 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:12.187188 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:12.203948 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d67e10-80af-4736-88ce-94cb3a22b0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:12.204581 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d67e10-80af-4736-88ce-94cb3a22b0d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:12.204828 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d67e10-80af-4736-88ce-94cb3a22b0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:12.205096 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d67e10-80af-4736-88ce-94cb3a22b0d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:12.206036 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d67e10-80af-4736-88ce-94cb3a22b0d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:12.208630 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d67e10-80af-4736-88ce-94cb3a22b0d5 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:12.219531 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1335/2674] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:35:12.224722 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce97d588-07d9-462c-bdd5-e5fc7a4eae21 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:12.235812 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce97d588-07d9-462c-bdd5-e5fc7a4eae21 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:12.241508 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e944c-81a8-42f0-aab5-8d31138655ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:12.242280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e944c-81a8-42f0-aab5-8d31138655ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:12.242554 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e944c-81a8-42f0-aab5-8d31138655ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:12.242826 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e944c-81a8-42f0-aab5-8d31138655ee admin admin] PATH_INFO: `/v3/projects/c205060c98094a2c9969cc9ba712edb3/users/a2966ff64c454a82a88f5349de3b2f98/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:12.244991 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e944c-81a8-42f0-aab5-8d31138655ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=c205060c98094a2c9969cc9ba712edb3, user_id=a2966ff64c454a82a88f5349de3b2f98, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:12.247627 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e944c-81a8-42f0-aab5-8d31138655ee admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:12.258550 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1340/2675] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:35:12 2022] PUT /identity/v3/projects/c205060c98094a2c9969cc9ba712edb3/users/a2966ff64c454a82a88f5349de3b2f98/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:35:12.273013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1dc008-6492-4187-be1a-5cb4c2f8c1aa None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:12.273487 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1dc008-6492-4187-be1a-5cb4c2f8c1aa None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:12.273783 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1dc008-6492-4187-be1a-5cb4c2f8c1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:12.305798 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-da1dc008-6492-4187-be1a-5cb4c2f8c1aa None None] MFA Rules not processed for user `a2966ff64c454a82a88f5349de3b2f98`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:12.307093 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-da1dc008-6492-4187-be1a-5cb4c2f8c1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:12.376173 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1336/2676] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:12 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:35:12.390202 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e944c-81a8-42f0-aab5-8d31138655ee admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:12.392184 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-cc9e944c-81a8-42f0-aab5-8d31138655ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:12.453042 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e944c-81a8-42f0-aab5-8d31138655ee admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:12.457861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b96e03-53ae-4c05-b79a-be1d996ce4fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:12.458543 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b96e03-53ae-4c05-b79a-be1d996ce4fd service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:12.459202 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b96e03-53ae-4c05-b79a-be1d996ce4fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:12.459202 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b96e03-53ae-4c05-b79a-be1d996ce4fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:12.472839 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b96e03-53ae-4c05-b79a-be1d996ce4fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:12.474392 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b96e03-53ae-4c05-b79a-be1d996ce4fd service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:12.484872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1341/2677] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:35:12 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:35:12.574107 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1dc008-6492-4187-be1a-5cb4c2f8c1aa None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:12.587689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1dc008-6492-4187-be1a-5cb4c2f8c1aa None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:12.594282 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8dc2a1-f18d-43ba-a33c-625585d58309 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:12.595143 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8dc2a1-f18d-43ba-a33c-625585d58309 service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:12.595449 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8dc2a1-f18d-43ba-a33c-625585d58309 service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:12.595759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8dc2a1-f18d-43ba-a33c-625585d58309 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:12.670313 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8dc2a1-f18d-43ba-a33c-625585d58309 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:12.671809 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8dc2a1-f18d-43ba-a33c-625585d58309 service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:12.681025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1337/2678] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:35:12 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:35:14.208572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b96e03-53ae-4c05-b79a-be1d996ce4fd service cinder] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:14.210299 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a8b96e03-53ae-4c05-b79a-be1d996ce4fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:14.231940 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:14.280465 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b96e03-53ae-4c05-b79a-be1d996ce4fd service cinder] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:14.284949 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8398e2-0e1d-4bbf-ba08-7ee07569a631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:14.285590 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8398e2-0e1d-4bbf-ba08-7ee07569a631 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:14.285868 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8398e2-0e1d-4bbf-ba08-7ee07569a631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:14.286139 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8398e2-0e1d-4bbf-ba08-7ee07569a631 admin admin] PATH_INFO: `/v3/users/a866b78df6b542c08bdec06232c6edd0` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:14.287286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8398e2-0e1d-4bbf-ba08-7ee07569a631 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a866b78df6b542c08bdec06232c6edd0)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:14.289231 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8398e2-0e1d-4bbf-ba08-7ee07569a631 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:14.338959 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ce8398e2-0e1d-4bbf-ba08-7ee07569a631 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a866b78df6b542c08bdec06232c6edd0'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:14.339603 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ce8398e2-0e1d-4bbf-ba08-7ee07569a631 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a866b78df6b542c08bdec06232c6edd0'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:14.347664 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ce8398e2-0e1d-4bbf-ba08-7ee07569a631 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a866b78df6b542c08bdec06232c6edd0'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:14.360109 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ce8398e2-0e1d-4bbf-ba08-7ee07569a631 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a866b78df6b542c08bdec06232c6edd0'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:14.366795 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-ce8398e2-0e1d-4bbf-ba08-7ee07569a631 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a866b78df6b542c08bdec06232c6edd0'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:14.373160 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1342/2679] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:35:14 2022] DELETE /identity/v3/users/a866b78df6b542c08bdec06232c6edd0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:35:14.377827 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018cf5e-307b-4427-9b78-b69c54d06830 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:14.378337 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018cf5e-307b-4427-9b78-b69c54d06830 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:14.378455 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018cf5e-307b-4427-9b78-b69c54d06830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:14.399685 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-3018cf5e-307b-4427-9b78-b69c54d06830 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:14.401280 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-3018cf5e-307b-4427-9b78-b69c54d06830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:14.466497 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1338/2680] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:35:14.476956 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8398e2-0e1d-4bbf-ba08-7ee07569a631 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:14.478688 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-ce8398e2-0e1d-4bbf-ba08-7ee07569a631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:14.537097 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8398e2-0e1d-4bbf-ba08-7ee07569a631 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:14.541105 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1b95d-d8b9-467e-80f3-cf6443036801 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:14.541715 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1b95d-d8b9-467e-80f3-cf6443036801 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:14.541992 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1b95d-d8b9-467e-80f3-cf6443036801 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:14.542264 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1b95d-d8b9-467e-80f3-cf6443036801 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:14.551109 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a1b95d-d8b9-467e-80f3-cf6443036801 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:14.552583 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a1b95d-d8b9-467e-80f3-cf6443036801 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:14.563066 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1343/2681] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:35:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:35:14.842601 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018cf5e-307b-4427-9b78-b69c54d06830 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:14.844284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-3018cf5e-307b-4427-9b78-b69c54d06830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:14.864501 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018cf5e-307b-4427-9b78-b69c54d06830 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:14.874242 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57d029-2d87-4ede-a67e-1a459b6ae4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:14.874899 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57d029-2d87-4ede-a67e-1a459b6ae4cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:14.875176 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57d029-2d87-4ede-a67e-1a459b6ae4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:14.875443 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57d029-2d87-4ede-a67e-1a459b6ae4cc admin admin] PATH_INFO: `/v3/projects/5dfe5e252c81484d93623b605e48746e` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:14.876752 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57d029-2d87-4ede-a67e-1a459b6ae4cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dfe5e252c81484d93623b605e48746e)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:14.878803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57d029-2d87-4ede-a67e-1a459b6ae4cc admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:14.880472 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:14.929761 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-2d57d029-2d87-4ede-a67e-1a459b6ae4cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dfe5e252c81484d93623b605e48746e', 'request_id': 'req-2d57d029-2d87-4ede-a67e-1a459b6ae4cc'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:14.936450 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-2d57d029-2d87-4ede-a67e-1a459b6ae4cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dfe5e252c81484d93623b605e48746e', 'request_id': 'req-2d57d029-2d87-4ede-a67e-1a459b6ae4cc'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:14.946012 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-2d57d029-2d87-4ede-a67e-1a459b6ae4cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dfe5e252c81484d93623b605e48746e', 'request_id': 'req-2d57d029-2d87-4ede-a67e-1a459b6ae4cc'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:14.960329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-2d57d029-2d87-4ede-a67e-1a459b6ae4cc admin admin] The token cache is being invalidate because project 5dfe5e252c81484d93623b605e48746e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:35:14.960547 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-2d57d029-2d87-4ede-a67e-1a459b6ae4cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:14.961623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-2d57d029-2d87-4ede-a67e-1a459b6ae4cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:14.964460 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1339/2682] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:35:14 2022] DELETE /identity/v3/projects/5dfe5e252c81484d93623b605e48746e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:35:14.970519 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be623fb7-14c9-49b9-ac2c-f94a3b3bdacf None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:14.970921 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be623fb7-14c9-49b9-ac2c-f94a3b3bdacf None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:14.971318 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be623fb7-14c9-49b9-ac2c-f94a3b3bdacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:14.997086 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-be623fb7-14c9-49b9-ac2c-f94a3b3bdacf None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:14.998446 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-be623fb7-14c9-49b9-ac2c-f94a3b3bdacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:15.033642 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:15.063065 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1344/2683] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:35:15.070956 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c758477-f107-45d6-b948-d01a3f4eb9b6 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:15.071246 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c758477-f107-45d6-b948-d01a3f4eb9b6 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:15.071558 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c758477-f107-45d6-b948-d01a3f4eb9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:15.092985 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-5c758477-f107-45d6-b948-d01a3f4eb9b6 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:15.094344 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-5c758477-f107-45d6-b948-d01a3f4eb9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:15.103564 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1340/2684] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:35:15.108270 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be623fb7-14c9-49b9-ac2c-f94a3b3bdacf None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:15.119215 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be623fb7-14c9-49b9-ac2c-f94a3b3bdacf None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:15.124095 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759c9a0-2efd-423d-94e6-4ec8f01e167c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:15.124796 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759c9a0-2efd-423d-94e6-4ec8f01e167c admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:15.125170 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759c9a0-2efd-423d-94e6-4ec8f01e167c admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:15.125506 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759c9a0-2efd-423d-94e6-4ec8f01e167c admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:15.126224 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f759c9a0-2efd-423d-94e6-4ec8f01e167c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:35:15.126593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f759c9a0-2efd-423d-94e6-4ec8f01e167c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:15.128237 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f759c9a0-2efd-423d-94e6-4ec8f01e167c admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:15.141017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1345/2685] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:35:15.145530 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c758477-f107-45d6-b948-d01a3f4eb9b6 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:15.155523 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c758477-f107-45d6-b948-d01a3f4eb9b6 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:15.160784 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac493c19-f76e-4cfb-8a9c-ed12cb6d3bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:15.161427 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac493c19-f76e-4cfb-8a9c-ed12cb6d3bdd admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:15.161702 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac493c19-f76e-4cfb-8a9c-ed12cb6d3bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:15.161973 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac493c19-f76e-4cfb-8a9c-ed12cb6d3bdd admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:15.162598 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac493c19-f76e-4cfb-8a9c-ed12cb6d3bdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:15.164566 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac493c19-f76e-4cfb-8a9c-ed12cb6d3bdd admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:15.192989 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1341/2686] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:35:15 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:35:15.197968 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759c9a0-2efd-423d-94e6-4ec8f01e167c admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:15.209181 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759c9a0-2efd-423d-94e6-4ec8f01e167c admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:15.275394 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6309ac9a-7d1b-493d-8e39-cabb7b696633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:15.276070 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309ac9a-7d1b-493d-8e39-cabb7b696633 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:15.276731 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309ac9a-7d1b-493d-8e39-cabb7b696633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:15.276731 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309ac9a-7d1b-493d-8e39-cabb7b696633 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:15.277278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6309ac9a-7d1b-493d-8e39-cabb7b696633 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:15.279292 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6309ac9a-7d1b-493d-8e39-cabb7b696633 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:15.306320 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1346/2687] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:35:15 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:35:15.307954 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac493c19-f76e-4cfb-8a9c-ed12cb6d3bdd admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:15.318412 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac493c19-f76e-4cfb-8a9c-ed12cb6d3bdd admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:15.323654 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82a55f-2a26-4b77-99d7-96533bf0da11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:15.324439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82a55f-2a26-4b77-99d7-96533bf0da11 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:15.324719 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82a55f-2a26-4b77-99d7-96533bf0da11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:15.325021 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82a55f-2a26-4b77-99d7-96533bf0da11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:15.325639 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b82a55f-2a26-4b77-99d7-96533bf0da11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:15.327252 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b82a55f-2a26-4b77-99d7-96533bf0da11 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:15.351295 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6309ac9a-7d1b-493d-8e39-cabb7b696633 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:15.352155 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1342/2688] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:35:15.362108 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6309ac9a-7d1b-493d-8e39-cabb7b696633 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:15.366875 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db989da2-bbb9-44f5-9b5c-12e0095f4d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:15.367487 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db989da2-bbb9-44f5-9b5c-12e0095f4d39 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:15.367751 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db989da2-bbb9-44f5-9b5c-12e0095f4d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:15.368087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db989da2-bbb9-44f5-9b5c-12e0095f4d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:15.369196 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db989da2-bbb9-44f5-9b5c-12e0095f4d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:15.370424 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db989da2-bbb9-44f5-9b5c-12e0095f4d39 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:15.382758 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1347/2689] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:35:15.386886 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82a55f-2a26-4b77-99d7-96533bf0da11 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:15.397565 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82a55f-2a26-4b77-99d7-96533bf0da11 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:15.402862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11377843-042f-4d3b-93ca-23edf555558a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:15.403532 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11377843-042f-4d3b-93ca-23edf555558a admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:15.403814 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11377843-042f-4d3b-93ca-23edf555558a admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:15.404135 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11377843-042f-4d3b-93ca-23edf555558a admin admin] PATH_INFO: `/v3/projects/2322d93bbb524167b9b01ae0da6f5586/users/def1ef82f373417aad12aaf695f1f1c8/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:15.419549 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11377843-042f-4d3b-93ca-23edf555558a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2322d93bbb524167b9b01ae0da6f5586, user_id=def1ef82f373417aad12aaf695f1f1c8, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:15.422347 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11377843-042f-4d3b-93ca-23edf555558a admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:15.436972 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1343/2690] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:35:15 2022] PUT /identity/v3/projects/2322d93bbb524167b9b01ae0da6f5586/users/def1ef82f373417aad12aaf695f1f1c8/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:35:15.448989 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085d682-2ac3-402c-91d2-7ed9b63fe3fb None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:15.449305 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085d682-2ac3-402c-91d2-7ed9b63fe3fb None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:15.449579 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085d682-2ac3-402c-91d2-7ed9b63fe3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:15.477663 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-a085d682-2ac3-402c-91d2-7ed9b63fe3fb None None] MFA Rules not processed for user `def1ef82f373417aad12aaf695f1f1c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:15.478793 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a085d682-2ac3-402c-91d2-7ed9b63fe3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:15.553268 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1348/2691] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:35:15.565459 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11377843-042f-4d3b-93ca-23edf555558a admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:15.567577 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-11377843-042f-4d3b-93ca-23edf555558a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:15.631361 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11377843-042f-4d3b-93ca-23edf555558a admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:15.635662 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5d59a-3b03-46c4-a4eb-0370294ac2fa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f1bf93cff6c46e0801bb827203cb5ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:15.636486 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5d59a-3b03-46c4-a4eb-0370294ac2fa service glance] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:15.636771 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5d59a-3b03-46c4-a4eb-0370294ac2fa service glance] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:15.637054 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5d59a-3b03-46c4-a4eb-0370294ac2fa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:15.645612 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5d59a-3b03-46c4-a4eb-0370294ac2fa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:15.647075 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5d59a-3b03-46c4-a4eb-0370294ac2fa service glance] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:15.656022 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1344/2692] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:35:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:35:25.550287 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=82,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:25.651957 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=77,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:27.787630 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085d682-2ac3-402c-91d2-7ed9b63fe3fb None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:27.789225 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a085d682-2ac3-402c-91d2-7ed9b63fe3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:27.920682 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085d682-2ac3-402c-91d2-7ed9b63fe3fb None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:27.926485 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45de963-638f-4c0f-a8e8-69117fb347cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:27.927120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45de963-638f-4c0f-a8e8-69117fb347cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:27.927401 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45de963-638f-4c0f-a8e8-69117fb347cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:27.927674 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45de963-638f-4c0f-a8e8-69117fb347cf admin admin] PATH_INFO: `/v3/users/702a25fbad0e40a385de9f5c2821fe13` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:27.928829 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45de963-638f-4c0f-a8e8-69117fb347cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=702a25fbad0e40a385de9f5c2821fe13)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:27.930782 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45de963-638f-4c0f-a8e8-69117fb347cf admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:27.982442 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a45de963-638f-4c0f-a8e8-69117fb347cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '702a25fbad0e40a385de9f5c2821fe13'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:27.983106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a45de963-638f-4c0f-a8e8-69117fb347cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '702a25fbad0e40a385de9f5c2821fe13'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:27.992289 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a45de963-638f-4c0f-a8e8-69117fb347cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '702a25fbad0e40a385de9f5c2821fe13'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:28.005251 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a45de963-638f-4c0f-a8e8-69117fb347cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '702a25fbad0e40a385de9f5c2821fe13'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:28.012296 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a45de963-638f-4c0f-a8e8-69117fb347cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '702a25fbad0e40a385de9f5c2821fe13'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:28.018559 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1349/2693] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:35:27 2022] DELETE /identity/v3/users/702a25fbad0e40a385de9f5c2821fe13 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:35:28.023254 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a46493-db58-4d56-9414-f486a6dcb0a9 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:28.023776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a46493-db58-4d56-9414-f486a6dcb0a9 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:28.023776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a46493-db58-4d56-9414-f486a6dcb0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:28.053769 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-f5a46493-db58-4d56-9414-f486a6dcb0a9 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:28.054964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-f5a46493-db58-4d56-9414-f486a6dcb0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:28.123094 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1345/2694] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:35:28.133148 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45de963-638f-4c0f-a8e8-69117fb347cf admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:28.134718 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a45de963-638f-4c0f-a8e8-69117fb347cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:28.157452 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:28.211639 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45de963-638f-4c0f-a8e8-69117fb347cf admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:28.215303 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5889340-4b6e-4a5a-912c-dba21a935ea5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:28.215925 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5889340-4b6e-4a5a-912c-dba21a935ea5 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:28.216212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5889340-4b6e-4a5a-912c-dba21a935ea5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:28.216483 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5889340-4b6e-4a5a-912c-dba21a935ea5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:28.226806 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5889340-4b6e-4a5a-912c-dba21a935ea5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:28.228285 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5889340-4b6e-4a5a-912c-dba21a935ea5 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:28.238155 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1350/2695] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:35:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:35:28.530629 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a46493-db58-4d56-9414-f486a6dcb0a9 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:28.532415 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-f5a46493-db58-4d56-9414-f486a6dcb0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:28.545596 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a46493-db58-4d56-9414-f486a6dcb0a9 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:28.550937 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa1d2c-6788-4bf8-a392-52abdd400c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:28.551581 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa1d2c-6788-4bf8-a392-52abdd400c08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:28.551863 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa1d2c-6788-4bf8-a392-52abdd400c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:28.552166 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa1d2c-6788-4bf8-a392-52abdd400c08 admin admin] PATH_INFO: `/v3/projects/4753c87e6a6b49cd9fde2a7350afab04` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:28.553304 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa1d2c-6788-4bf8-a392-52abdd400c08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4753c87e6a6b49cd9fde2a7350afab04)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:28.555224 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa1d2c-6788-4bf8-a392-52abdd400c08 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:28.602885 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-82fa1d2c-6788-4bf8-a392-52abdd400c08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4753c87e6a6b49cd9fde2a7350afab04', 'request_id': 'req-82fa1d2c-6788-4bf8-a392-52abdd400c08'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:28.613749 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-82fa1d2c-6788-4bf8-a392-52abdd400c08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4753c87e6a6b49cd9fde2a7350afab04', 'request_id': 'req-82fa1d2c-6788-4bf8-a392-52abdd400c08'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:28.624166 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-82fa1d2c-6788-4bf8-a392-52abdd400c08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4753c87e6a6b49cd9fde2a7350afab04', 'request_id': 'req-82fa1d2c-6788-4bf8-a392-52abdd400c08'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:28.637273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-82fa1d2c-6788-4bf8-a392-52abdd400c08 admin admin] The token cache is being invalidate because project 4753c87e6a6b49cd9fde2a7350afab04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:35:28.637719 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-82fa1d2c-6788-4bf8-a392-52abdd400c08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:28.638807 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-82fa1d2c-6788-4bf8-a392-52abdd400c08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:28.642043 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1346/2696] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:35:28 2022] DELETE /identity/v3/projects/4753c87e6a6b49cd9fde2a7350afab04 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:35:28.646800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5889340-4b6e-4a5a-912c-dba21a935ea5 service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:28.660550 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5889340-4b6e-4a5a-912c-dba21a935ea5 service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:28.725385 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae5aee-e5f1-4097-b009-3555208bf978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:28.726409 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae5aee-e5f1-4097-b009-3555208bf978 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:28.726766 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae5aee-e5f1-4097-b009-3555208bf978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:28.727036 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae5aee-e5f1-4097-b009-3555208bf978 admin admin] PATH_INFO: `/v3/users/a2966ff64c454a82a88f5349de3b2f98` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:28.728312 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae5aee-e5f1-4097-b009-3555208bf978 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2966ff64c454a82a88f5349de3b2f98)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:28.730250 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae5aee-e5f1-4097-b009-3555208bf978 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:28.783899 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-54ae5aee-e5f1-4097-b009-3555208bf978 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2966ff64c454a82a88f5349de3b2f98'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:28.784784 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-54ae5aee-e5f1-4097-b009-3555208bf978 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2966ff64c454a82a88f5349de3b2f98'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:28.799595 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-54ae5aee-e5f1-4097-b009-3555208bf978 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2966ff64c454a82a88f5349de3b2f98'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:28.821681 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-54ae5aee-e5f1-4097-b009-3555208bf978 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2966ff64c454a82a88f5349de3b2f98'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:28.835486 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-54ae5aee-e5f1-4097-b009-3555208bf978 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2966ff64c454a82a88f5349de3b2f98'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:28.837436 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1351/2697] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:35:28 2022] DELETE /identity/v3/users/a2966ff64c454a82a88f5349de3b2f98 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:35:29.122204 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa1d2c-6788-4bf8-a392-52abdd400c08 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:29.123992 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-82fa1d2c-6788-4bf8-a392-52abdd400c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:29.138520 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa1d2c-6788-4bf8-a392-52abdd400c08 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:29.206177 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258c8502-bbcb-45c0-a4de-8c63a19e9656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:29.206938 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c8502-bbcb-45c0-a4de-8c63a19e9656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:29.207227 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c8502-bbcb-45c0-a4de-8c63a19e9656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:29.207504 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c8502-bbcb-45c0-a4de-8c63a19e9656 admin admin] PATH_INFO: `/v3/projects/c205060c98094a2c9969cc9ba712edb3` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:29.208746 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258c8502-bbcb-45c0-a4de-8c63a19e9656 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c205060c98094a2c9969cc9ba712edb3)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:29.210722 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258c8502-bbcb-45c0-a4de-8c63a19e9656 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:29.234529 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:29.262319 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-258c8502-bbcb-45c0-a4de-8c63a19e9656 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c205060c98094a2c9969cc9ba712edb3', 'request_id': 'req-258c8502-bbcb-45c0-a4de-8c63a19e9656'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:29.269806 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-258c8502-bbcb-45c0-a4de-8c63a19e9656 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c205060c98094a2c9969cc9ba712edb3', 'request_id': 'req-258c8502-bbcb-45c0-a4de-8c63a19e9656'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:29.277797 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-258c8502-bbcb-45c0-a4de-8c63a19e9656 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c205060c98094a2c9969cc9ba712edb3', 'request_id': 'req-258c8502-bbcb-45c0-a4de-8c63a19e9656'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:29.289903 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-258c8502-bbcb-45c0-a4de-8c63a19e9656 admin admin] The token cache is being invalidate because project c205060c98094a2c9969cc9ba712edb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:35:29.290220 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-258c8502-bbcb-45c0-a4de-8c63a19e9656 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:29.290855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-258c8502-bbcb-45c0-a4de-8c63a19e9656 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:29.292439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1347/2698] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:35:29 2022] DELETE /identity/v3/projects/c205060c98094a2c9969cc9ba712edb3 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:35:29.297539 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb633c7-46cd-4611-89a9-d538a911968b None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:29.297960 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb633c7-46cd-4611-89a9-d538a911968b None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:29.298306 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb633c7-46cd-4611-89a9-d538a911968b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:29.345076 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:29.367754 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-bbb633c7-46cd-4611-89a9-d538a911968b None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:29.369038 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-bbb633c7-46cd-4611-89a9-d538a911968b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:29.429087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1352/2699] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:35:29.438088 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e322355-6413-4f37-ba9f-7e2b56e3da4f None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:29.438379 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e322355-6413-4f37-ba9f-7e2b56e3da4f None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:29.438657 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e322355-6413-4f37-ba9f-7e2b56e3da4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:29.459885 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-9e322355-6413-4f37-ba9f-7e2b56e3da4f None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:29.461212 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-9e322355-6413-4f37-ba9f-7e2b56e3da4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:29.470338 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1348/2700] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:35:29.475203 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb633c7-46cd-4611-89a9-d538a911968b None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:29.485701 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb633c7-46cd-4611-89a9-d538a911968b None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:29.490487 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa580bf-c3df-464b-bdba-d8157f0ad347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:29.491093 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa580bf-c3df-464b-bdba-d8157f0ad347 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:29.491373 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa580bf-c3df-464b-bdba-d8157f0ad347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:29.491645 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa580bf-c3df-464b-bdba-d8157f0ad347 admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:29.492323 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa580bf-c3df-464b-bdba-d8157f0ad347 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:35:29.492658 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa580bf-c3df-464b-bdba-d8157f0ad347 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:29.494395 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa580bf-c3df-464b-bdba-d8157f0ad347 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:29.506556 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1353/2701] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:35:29.511113 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e322355-6413-4f37-ba9f-7e2b56e3da4f None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:29.521451 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e322355-6413-4f37-ba9f-7e2b56e3da4f None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:29.527041 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2f4e5-488e-4472-8d50-5e1ebf71c820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:29.527652 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2f4e5-488e-4472-8d50-5e1ebf71c820 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:29.527989 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2f4e5-488e-4472-8d50-5e1ebf71c820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:29.528232 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2f4e5-488e-4472-8d50-5e1ebf71c820 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:29.529086 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a2f4e5-488e-4472-8d50-5e1ebf71c820 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:29.531087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a2f4e5-488e-4472-8d50-5e1ebf71c820 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:29.561250 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1349/2702] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:35:29 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:35:29.565560 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa580bf-c3df-464b-bdba-d8157f0ad347 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:29.576332 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa580bf-c3df-464b-bdba-d8157f0ad347 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:29.636855 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b99130-5eaf-4b86-bba1-1cd211bdd59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:29.637472 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b99130-5eaf-4b86-bba1-1cd211bdd59d admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:29.637754 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b99130-5eaf-4b86-bba1-1cd211bdd59d admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:29.638029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b99130-5eaf-4b86-bba1-1cd211bdd59d admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:29.638671 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b99130-5eaf-4b86-bba1-1cd211bdd59d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:29.640663 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b99130-5eaf-4b86-bba1-1cd211bdd59d admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:29.663978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1354/2703] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:35:29 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:35:29.668853 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2f4e5-488e-4472-8d50-5e1ebf71c820 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:29.680175 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2f4e5-488e-4472-8d50-5e1ebf71c820 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:29.685269 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef8bb7a-eae3-467f-8a29-05047d4bee1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:29.685893 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8bb7a-eae3-467f-8a29-05047d4bee1d admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:29.686165 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8bb7a-eae3-467f-8a29-05047d4bee1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:29.686430 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8bb7a-eae3-467f-8a29-05047d4bee1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:29.687071 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef8bb7a-eae3-467f-8a29-05047d4bee1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:29.688709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef8bb7a-eae3-467f-8a29-05047d4bee1d admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:29.699032 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1350/2704] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:35:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:35:29.703189 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b99130-5eaf-4b86-bba1-1cd211bdd59d admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:29.713860 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b99130-5eaf-4b86-bba1-1cd211bdd59d admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:29.718857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe56a735-3ad2-4685-87c5-7c0c19eb6cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:29.719464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe56a735-3ad2-4685-87c5-7c0c19eb6cac admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:29.719746 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe56a735-3ad2-4685-87c5-7c0c19eb6cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:29.720051 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe56a735-3ad2-4685-87c5-7c0c19eb6cac admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:29.720741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe56a735-3ad2-4685-87c5-7c0c19eb6cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:29.722323 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe56a735-3ad2-4685-87c5-7c0c19eb6cac admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:29.733476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1355/2705] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:35:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:35:29.738423 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef8bb7a-eae3-467f-8a29-05047d4bee1d admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:29.751393 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef8bb7a-eae3-467f-8a29-05047d4bee1d admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:29.757502 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c6712-255a-43be-9644-94b9604252f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:29.758196 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c6712-255a-43be-9644-94b9604252f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:29.758469 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c6712-255a-43be-9644-94b9604252f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:29.758734 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c6712-255a-43be-9644-94b9604252f7 admin admin] PATH_INFO: `/v3/projects/2694b3f6dc874e8593b2a2dd3c5597d5/users/8b488130acf14868b7d7e3b37d5c2708/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:29.778433 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93c6712-255a-43be-9644-94b9604252f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2694b3f6dc874e8593b2a2dd3c5597d5, user_id=8b488130acf14868b7d7e3b37d5c2708, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:29.782900 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93c6712-255a-43be-9644-94b9604252f7 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:29.799259 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1351/2706] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:35:29 2022] PUT /identity/v3/projects/2694b3f6dc874e8593b2a2dd3c5597d5/users/8b488130acf14868b7d7e3b37d5c2708/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:35:29.809758 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a2f2c-dcab-4aec-b454-39b2a25d6929 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:29.810238 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a2f2c-dcab-4aec-b454-39b2a25d6929 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:29.810363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a2f2c-dcab-4aec-b454-39b2a25d6929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:29.840772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-960a2f2c-dcab-4aec-b454-39b2a25d6929 None None] MFA Rules not processed for user `8b488130acf14868b7d7e3b37d5c2708`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:29.841992 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-960a2f2c-dcab-4aec-b454-39b2a25d6929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:29.902862 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:29.913305 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1356/2707] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:35:29.925449 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c6712-255a-43be-9644-94b9604252f7 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:29.927423 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-e93c6712-255a-43be-9644-94b9604252f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:29.989390 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c6712-255a-43be-9644-94b9604252f7 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:29.993263 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081ae30-8ae5-4c52-9387-b7eb1a37aa57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:29.993964 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081ae30-8ae5-4c52-9387-b7eb1a37aa57 service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:29.994432 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081ae30-8ae5-4c52-9387-b7eb1a37aa57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:29.994623 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081ae30-8ae5-4c52-9387-b7eb1a37aa57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:30.006873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081ae30-8ae5-4c52-9387-b7eb1a37aa57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:30.008456 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081ae30-8ae5-4c52-9387-b7eb1a37aa57 service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:30.024053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1352/2708] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:35:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:35:39.918831 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=2 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:40.015099 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=69,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:44.699095 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960a2f2c-dcab-4aec-b454-39b2a25d6929 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:44.700717 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-960a2f2c-dcab-4aec-b454-39b2a25d6929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:44.901114 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960a2f2c-dcab-4aec-b454-39b2a25d6929 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:44.905951 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1edd8-4482-4ca7-97ae-fd0dbb7f17ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:44.906588 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1edd8-4482-4ca7-97ae-fd0dbb7f17ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:44.906865 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1edd8-4482-4ca7-97ae-fd0dbb7f17ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:44.907133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1edd8-4482-4ca7-97ae-fd0dbb7f17ad admin admin] PATH_INFO: `/v3/users/def1ef82f373417aad12aaf695f1f1c8` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:44.908315 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d1edd8-4482-4ca7-97ae-fd0dbb7f17ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=def1ef82f373417aad12aaf695f1f1c8)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:44.910231 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d1edd8-4482-4ca7-97ae-fd0dbb7f17ad admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:44.963842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a8d1edd8-4482-4ca7-97ae-fd0dbb7f17ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'def1ef82f373417aad12aaf695f1f1c8'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:44.964432 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a8d1edd8-4482-4ca7-97ae-fd0dbb7f17ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'def1ef82f373417aad12aaf695f1f1c8'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:44.973491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a8d1edd8-4482-4ca7-97ae-fd0dbb7f17ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'def1ef82f373417aad12aaf695f1f1c8'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:44.986973 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a8d1edd8-4482-4ca7-97ae-fd0dbb7f17ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'def1ef82f373417aad12aaf695f1f1c8'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:44.994029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a8d1edd8-4482-4ca7-97ae-fd0dbb7f17ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'def1ef82f373417aad12aaf695f1f1c8'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:45.000471 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1357/2709] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:35:44 2022] DELETE /identity/v3/users/def1ef82f373417aad12aaf695f1f1c8 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:35:45.005214 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536df5e-5450-4b4f-9cfc-f38e4b1fd26b None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:45.005736 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536df5e-5450-4b4f-9cfc-f38e4b1fd26b None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:45.005832 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536df5e-5450-4b4f-9cfc-f38e4b1fd26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:45.025541 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-2536df5e-5450-4b4f-9cfc-f38e4b1fd26b None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:45.026750 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-2536df5e-5450-4b4f-9cfc-f38e4b1fd26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:45.090338 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1353/2710] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:35:45.102748 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1edd8-4482-4ca7-97ae-fd0dbb7f17ad admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:45.104571 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a8d1edd8-4482-4ca7-97ae-fd0dbb7f17ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:45.192368 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1edd8-4482-4ca7-97ae-fd0dbb7f17ad admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:45.196420 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799cb7d9-a0db-4271-9f01-90812a07e10b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:45.197029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cb7d9-a0db-4271-9f01-90812a07e10b service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:45.197307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cb7d9-a0db-4271-9f01-90812a07e10b service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:45.197573 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cb7d9-a0db-4271-9f01-90812a07e10b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:45.206626 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799cb7d9-a0db-4271-9f01-90812a07e10b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:45.208104 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799cb7d9-a0db-4271-9f01-90812a07e10b service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:45.217491 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1358/2711] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:35:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:35:45.743393 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536df5e-5450-4b4f-9cfc-f38e4b1fd26b None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:45.744967 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-2536df5e-5450-4b4f-9cfc-f38e4b1fd26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:45.760898 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536df5e-5450-4b4f-9cfc-f38e4b1fd26b None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:45.766159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3d76c-ab96-4d5f-abd6-2fdd598793a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:45.766866 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3d76c-ab96-4d5f-abd6-2fdd598793a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:45.767085 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3d76c-ab96-4d5f-abd6-2fdd598793a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:45.767370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3d76c-ab96-4d5f-abd6-2fdd598793a7 admin admin] PATH_INFO: `/v3/projects/2322d93bbb524167b9b01ae0da6f5586` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:45.768516 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3d76c-ab96-4d5f-abd6-2fdd598793a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2322d93bbb524167b9b01ae0da6f5586)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:45.770469 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3d76c-ab96-4d5f-abd6-2fdd598793a7 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:45.819900 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-4ea3d76c-ab96-4d5f-abd6-2fdd598793a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2322d93bbb524167b9b01ae0da6f5586', 'request_id': 'req-4ea3d76c-ab96-4d5f-abd6-2fdd598793a7'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:45.826089 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-4ea3d76c-ab96-4d5f-abd6-2fdd598793a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2322d93bbb524167b9b01ae0da6f5586', 'request_id': 'req-4ea3d76c-ab96-4d5f-abd6-2fdd598793a7'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:45.834599 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-4ea3d76c-ab96-4d5f-abd6-2fdd598793a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2322d93bbb524167b9b01ae0da6f5586', 'request_id': 'req-4ea3d76c-ab96-4d5f-abd6-2fdd598793a7'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:45.845112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-4ea3d76c-ab96-4d5f-abd6-2fdd598793a7 admin admin] The token cache is being invalidate because project 2322d93bbb524167b9b01ae0da6f5586 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:35:45.845410 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-4ea3d76c-ab96-4d5f-abd6-2fdd598793a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:45.846005 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-4ea3d76c-ab96-4d5f-abd6-2fdd598793a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:45.847476 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1354/2712] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:35:45 2022] DELETE /identity/v3/projects/2322d93bbb524167b9b01ae0da6f5586 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:35:45.852967 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c4be1-37c1-4560-91ac-ce93b5ed66a8 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:45.853464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c4be1-37c1-4560-91ac-ce93b5ed66a8 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:45.853580 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c4be1-37c1-4560-91ac-ce93b5ed66a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:45.861363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:45.875555 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-d28c4be1-37c1-4560-91ac-ce93b5ed66a8 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:45.877266 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-d28c4be1-37c1-4560-91ac-ce93b5ed66a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:45.942151 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1359/2713] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:35:45.950219 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed36e3-6d61-4885-942b-2451caaf9cd5 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:45.950513 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed36e3-6d61-4885-942b-2451caaf9cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:45.950793 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed36e3-6d61-4885-942b-2451caaf9cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:45.970698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-b6ed36e3-6d61-4885-942b-2451caaf9cd5 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:45.971907 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-b6ed36e3-6d61-4885-942b-2451caaf9cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:45.980564 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1355/2714] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:35:45.984869 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c4be1-37c1-4560-91ac-ce93b5ed66a8 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:45.995137 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c4be1-37c1-4560-91ac-ce93b5ed66a8 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:46.000555 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df876498-d10e-4985-848a-0a8c1b2ba79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:46.001158 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df876498-d10e-4985-848a-0a8c1b2ba79e admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:46.001437 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df876498-d10e-4985-848a-0a8c1b2ba79e admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:46.001707 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df876498-d10e-4985-848a-0a8c1b2ba79e admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:46.002391 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df876498-d10e-4985-848a-0a8c1b2ba79e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:35:46.002694 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df876498-d10e-4985-848a-0a8c1b2ba79e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:46.004319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df876498-d10e-4985-848a-0a8c1b2ba79e admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:46.016167 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1360/2715] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:35:46.021774 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed36e3-6d61-4885-942b-2451caaf9cd5 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:46.031451 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed36e3-6d61-4885-942b-2451caaf9cd5 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:46.036553 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f37ad6-5666-4c3b-a2ba-300237f08ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:46.037188 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f37ad6-5666-4c3b-a2ba-300237f08ad8 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:46.037469 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f37ad6-5666-4c3b-a2ba-300237f08ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:46.038098 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f37ad6-5666-4c3b-a2ba-300237f08ad8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:46.038411 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f37ad6-5666-4c3b-a2ba-300237f08ad8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:46.040380 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f37ad6-5666-4c3b-a2ba-300237f08ad8 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:46.070483 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1356/2716] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:35:46 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:35:46.074884 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df876498-d10e-4985-848a-0a8c1b2ba79e admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:46.085065 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df876498-d10e-4985-848a-0a8c1b2ba79e admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:46.152012 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0079c2-df02-4e0e-8b62-745e32a6dcba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:46.152638 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0079c2-df02-4e0e-8b62-745e32a6dcba admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:46.152914 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0079c2-df02-4e0e-8b62-745e32a6dcba admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:46.153203 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0079c2-df02-4e0e-8b62-745e32a6dcba admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:46.153809 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0079c2-df02-4e0e-8b62-745e32a6dcba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:46.155702 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0079c2-df02-4e0e-8b62-745e32a6dcba admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:46.179410 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1361/2717] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:35:46 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:35:46.186268 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f37ad6-5666-4c3b-a2ba-300237f08ad8 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:46.200965 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f37ad6-5666-4c3b-a2ba-300237f08ad8 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:46.205913 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb2809-0d46-487a-bc02-4f5147b0165d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:46.206516 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb2809-0d46-487a-bc02-4f5147b0165d admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:46.206799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb2809-0d46-487a-bc02-4f5147b0165d admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:46.207072 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb2809-0d46-487a-bc02-4f5147b0165d admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:46.207713 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb2809-0d46-487a-bc02-4f5147b0165d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:46.209359 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb2809-0d46-487a-bc02-4f5147b0165d admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:46.223699 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1357/2718] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:35:46.229439 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0079c2-df02-4e0e-8b62-745e32a6dcba admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:46.246504 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0079c2-df02-4e0e-8b62-745e32a6dcba admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:46.254533 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97233fc0-60c9-4749-a718-ddf2c23418a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:46.255390 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97233fc0-60c9-4749-a718-ddf2c23418a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:46.255759 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97233fc0-60c9-4749-a718-ddf2c23418a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:46.256156 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97233fc0-60c9-4749-a718-ddf2c23418a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:46.257185 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97233fc0-60c9-4749-a718-ddf2c23418a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:46.258862 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97233fc0-60c9-4749-a718-ddf2c23418a9 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:46.269538 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1362/2719] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:35:46.274219 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb2809-0d46-487a-bc02-4f5147b0165d admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:46.284720 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb2809-0d46-487a-bc02-4f5147b0165d admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:46.290066 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62bfb5a-eecf-4301-9d3b-0ebb26df74a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:46.290712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62bfb5a-eecf-4301-9d3b-0ebb26df74a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:46.290967 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62bfb5a-eecf-4301-9d3b-0ebb26df74a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:46.291271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62bfb5a-eecf-4301-9d3b-0ebb26df74a3 admin admin] PATH_INFO: `/v3/projects/de57031919624c4d8f5e3a2ddcb90bfd/users/1a73b8869b374f9d85f16b561a28061b/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:46.313269 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62bfb5a-eecf-4301-9d3b-0ebb26df74a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de57031919624c4d8f5e3a2ddcb90bfd, user_id=1a73b8869b374f9d85f16b561a28061b, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:46.315966 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62bfb5a-eecf-4301-9d3b-0ebb26df74a3 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:46.332850 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1358/2720] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:35:46 2022] PUT /identity/v3/projects/de57031919624c4d8f5e3a2ddcb90bfd/users/1a73b8869b374f9d85f16b561a28061b/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:35:46.343373 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66832ede-9856-4036-a432-695dd360715a None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:46.343664 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66832ede-9856-4036-a432-695dd360715a None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:46.343945 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66832ede-9856-4036-a432-695dd360715a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:46.374471 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-66832ede-9856-4036-a432-695dd360715a None None] MFA Rules not processed for user `1a73b8869b374f9d85f16b561a28061b`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:46.375689 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-66832ede-9856-4036-a432-695dd360715a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:46.446349 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:46.452379 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1363/2721] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:35:46.463784 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62bfb5a-eecf-4301-9d3b-0ebb26df74a3 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:46.466383 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-e62bfb5a-eecf-4301-9d3b-0ebb26df74a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:46.498158 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:46.584162 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62bfb5a-eecf-4301-9d3b-0ebb26df74a3 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:46.589631 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c1530-dbc8-4d44-9ec9-e89ec14a1133 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:46.590446 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c1530-dbc8-4d44-9ec9-e89ec14a1133 service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:46.590723 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c1530-dbc8-4d44-9ec9-e89ec14a1133 service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:46.590992 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c1530-dbc8-4d44-9ec9-e89ec14a1133 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:46.601873 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c1530-dbc8-4d44-9ec9-e89ec14a1133 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:46.603437 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c1530-dbc8-4d44-9ec9-e89ec14a1133 service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:46.615391 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1359/2722] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:35:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:35:50.939726 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66832ede-9856-4036-a432-695dd360715a None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:50.941362 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-66832ede-9856-4036-a432-695dd360715a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:51.013888 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66832ede-9856-4036-a432-695dd360715a None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:51.019506 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e5c92-783e-4708-970a-5753b760e3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:51.020199 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e5c92-783e-4708-970a-5753b760e3dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:51.020480 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e5c92-783e-4708-970a-5753b760e3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:51.020790 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e5c92-783e-4708-970a-5753b760e3dd admin admin] PATH_INFO: `/v3/users/8b488130acf14868b7d7e3b37d5c2708` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:51.022090 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403e5c92-783e-4708-970a-5753b760e3dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b488130acf14868b7d7e3b37d5c2708)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:51.024110 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403e5c92-783e-4708-970a-5753b760e3dd admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:51.080474 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-403e5c92-783e-4708-970a-5753b760e3dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b488130acf14868b7d7e3b37d5c2708'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:51.081111 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-403e5c92-783e-4708-970a-5753b760e3dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b488130acf14868b7d7e3b37d5c2708'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:51.089290 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-403e5c92-783e-4708-970a-5753b760e3dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b488130acf14868b7d7e3b37d5c2708'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:51.101058 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-403e5c92-783e-4708-970a-5753b760e3dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b488130acf14868b7d7e3b37d5c2708'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:51.108934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-403e5c92-783e-4708-970a-5753b760e3dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b488130acf14868b7d7e3b37d5c2708'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:51.115129 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1364/2723] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:35:50 2022] DELETE /identity/v3/users/8b488130acf14868b7d7e3b37d5c2708 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:35:51.119797 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041b245-739e-44f0-9036-b42647d8b5ee None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:51.120307 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041b245-739e-44f0-9036-b42647d8b5ee None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:51.120447 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041b245-739e-44f0-9036-b42647d8b5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:51.141292 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-6041b245-739e-44f0-9036-b42647d8b5ee None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:51.142517 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-6041b245-739e-44f0-9036-b42647d8b5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:51.203085 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1360/2724] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:35:51.213200 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e5c92-783e-4708-970a-5753b760e3dd admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:51.214781 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-403e5c92-783e-4708-970a-5753b760e3dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:51.273981 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e5c92-783e-4708-970a-5753b760e3dd admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:51.277942 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a55f9-de08-452e-9452-e16056a7a638 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:51.278549 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a55f9-de08-452e-9452-e16056a7a638 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:51.278825 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a55f9-de08-452e-9452-e16056a7a638 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:51.279094 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a55f9-de08-452e-9452-e16056a7a638 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:51.289082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425a55f9-de08-452e-9452-e16056a7a638 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:51.290507 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425a55f9-de08-452e-9452-e16056a7a638 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:51.300117 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1365/2725] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:35:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:35:51.583432 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041b245-739e-44f0-9036-b42647d8b5ee None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:51.585110 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-6041b245-739e-44f0-9036-b42647d8b5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:51.599280 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041b245-739e-44f0-9036-b42647d8b5ee None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:51.604812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ab001-5819-4ca9-be4b-43ee9a1f3f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:51.605506 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ab001-5819-4ca9-be4b-43ee9a1f3f26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:51.605724 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ab001-5819-4ca9-be4b-43ee9a1f3f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:51.605998 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ab001-5819-4ca9-be4b-43ee9a1f3f26 admin admin] PATH_INFO: `/v3/projects/2694b3f6dc874e8593b2a2dd3c5597d5` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:51.607436 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ab001-5819-4ca9-be4b-43ee9a1f3f26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2694b3f6dc874e8593b2a2dd3c5597d5)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:51.609556 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ab001-5819-4ca9-be4b-43ee9a1f3f26 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:51.659206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-857ab001-5819-4ca9-be4b-43ee9a1f3f26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2694b3f6dc874e8593b2a2dd3c5597d5', 'request_id': 'req-857ab001-5819-4ca9-be4b-43ee9a1f3f26'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:51.665756 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-857ab001-5819-4ca9-be4b-43ee9a1f3f26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2694b3f6dc874e8593b2a2dd3c5597d5', 'request_id': 'req-857ab001-5819-4ca9-be4b-43ee9a1f3f26'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:51.673877 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-857ab001-5819-4ca9-be4b-43ee9a1f3f26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2694b3f6dc874e8593b2a2dd3c5597d5', 'request_id': 'req-857ab001-5819-4ca9-be4b-43ee9a1f3f26'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:51.685297 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-857ab001-5819-4ca9-be4b-43ee9a1f3f26 admin admin] The token cache is being invalidate because project 2694b3f6dc874e8593b2a2dd3c5597d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:35:51.685483 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-857ab001-5819-4ca9-be4b-43ee9a1f3f26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:51.686198 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-857ab001-5819-4ca9-be4b-43ee9a1f3f26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:51.687838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1361/2726] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:35:51 2022] DELETE /identity/v3/projects/2694b3f6dc874e8593b2a2dd3c5597d5 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:35:51.692970 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27bf992-887f-493e-93f8-7fac835f8037 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:51.693223 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27bf992-887f-493e-93f8-7fac835f8037 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:51.693528 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27bf992-887f-493e-93f8-7fac835f8037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:51.713163 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-a27bf992-887f-493e-93f8-7fac835f8037 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:51.714512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a27bf992-887f-493e-93f8-7fac835f8037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:51.729861 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:51.781086 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1366/2727] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:35:51.788998 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd05bf-119d-4408-99e7-dae79ae20cc8 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:51.789286 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd05bf-119d-4408-99e7-dae79ae20cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:51.789564 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd05bf-119d-4408-99e7-dae79ae20cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:51.811524 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-41fd05bf-119d-4408-99e7-dae79ae20cc8 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:51.812786 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-41fd05bf-119d-4408-99e7-dae79ae20cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:51.821206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1362/2728] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:35:51.825269 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27bf992-887f-493e-93f8-7fac835f8037 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:51.835433 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27bf992-887f-493e-93f8-7fac835f8037 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:51.840410 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a28cd-d8d1-4e2b-a81f-7f86faf18343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:51.842363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a28cd-d8d1-4e2b-a81f-7f86faf18343 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:51.842363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a28cd-d8d1-4e2b-a81f-7f86faf18343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:51.842363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a28cd-d8d1-4e2b-a81f-7f86faf18343 admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:51.842363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a28cd-d8d1-4e2b-a81f-7f86faf18343 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:35:51.842769 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a28cd-d8d1-4e2b-a81f-7f86faf18343 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:51.844110 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a28cd-d8d1-4e2b-a81f-7f86faf18343 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:51.856434 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1367/2729] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:35:51.860461 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd05bf-119d-4408-99e7-dae79ae20cc8 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:51.869903 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd05bf-119d-4408-99e7-dae79ae20cc8 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:51.874647 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2505a-71a9-4039-b6fd-1b85765bf00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:51.875250 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2505a-71a9-4039-b6fd-1b85765bf00d admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:51.875541 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2505a-71a9-4039-b6fd-1b85765bf00d admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:51.875811 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2505a-71a9-4039-b6fd-1b85765bf00d admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:51.876498 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2505a-71a9-4039-b6fd-1b85765bf00d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:51.878413 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2505a-71a9-4039-b6fd-1b85765bf00d admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:51.901502 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:51.912081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1363/2730] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:35:51 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:35:51.913692 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a28cd-d8d1-4e2b-a81f-7f86faf18343 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:51.924340 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a28cd-d8d1-4e2b-a81f-7f86faf18343 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:51.982888 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71300ee9-7849-42fb-80e0-eb00bf48d8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:51.983652 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71300ee9-7849-42fb-80e0-eb00bf48d8aa admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:51.983945 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71300ee9-7849-42fb-80e0-eb00bf48d8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:51.984220 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71300ee9-7849-42fb-80e0-eb00bf48d8aa admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:51.984874 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71300ee9-7849-42fb-80e0-eb00bf48d8aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:51.986811 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71300ee9-7849-42fb-80e0-eb00bf48d8aa admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:52.015817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1368/2731] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:35:51 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:35:52.017382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2505a-71a9-4039-b6fd-1b85765bf00d admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:52.028819 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2505a-71a9-4039-b6fd-1b85765bf00d admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:52.033739 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b96706e-3e92-4afd-b284-583b24faf0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:52.034389 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b96706e-3e92-4afd-b284-583b24faf0ed admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:52.034666 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b96706e-3e92-4afd-b284-583b24faf0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:52.034939 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b96706e-3e92-4afd-b284-583b24faf0ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:52.035606 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b96706e-3e92-4afd-b284-583b24faf0ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:52.037244 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b96706e-3e92-4afd-b284-583b24faf0ed admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:52.047891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1364/2732] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:35:52.052426 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71300ee9-7849-42fb-80e0-eb00bf48d8aa admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:52.062920 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71300ee9-7849-42fb-80e0-eb00bf48d8aa admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:52.067664 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f50cd3-7fb1-4e03-a6b2-81f761dbf60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:52.068288 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f50cd3-7fb1-4e03-a6b2-81f761dbf60a admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:52.068570 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f50cd3-7fb1-4e03-a6b2-81f761dbf60a admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:52.068840 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f50cd3-7fb1-4e03-a6b2-81f761dbf60a admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:52.069482 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f50cd3-7fb1-4e03-a6b2-81f761dbf60a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:52.071224 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f50cd3-7fb1-4e03-a6b2-81f761dbf60a admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:52.083034 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1369/2733] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:35:52.087505 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b96706e-3e92-4afd-b284-583b24faf0ed admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:52.098180 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b96706e-3e92-4afd-b284-583b24faf0ed admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:52.103962 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe84da-b5b5-41a2-8852-a27b107d69ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:52.104617 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe84da-b5b5-41a2-8852-a27b107d69ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:52.104895 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe84da-b5b5-41a2-8852-a27b107d69ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:52.105163 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe84da-b5b5-41a2-8852-a27b107d69ba admin admin] PATH_INFO: `/v3/projects/191c6b14aa0d4277920cd57df4d24c95/users/3561c92204194888a26f9390904ede55/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:52.120653 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe84da-b5b5-41a2-8852-a27b107d69ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=191c6b14aa0d4277920cd57df4d24c95, user_id=3561c92204194888a26f9390904ede55, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:52.123230 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe84da-b5b5-41a2-8852-a27b107d69ba admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:52.137962 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1365/2734] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:35:52 2022] PUT /identity/v3/projects/191c6b14aa0d4277920cd57df4d24c95/users/3561c92204194888a26f9390904ede55/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:35:52.149946 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7ac83-5332-4c54-947b-ca9b7e215b2b None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:52.150339 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7ac83-5332-4c54-947b-ca9b7e215b2b None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:52.150574 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7ac83-5332-4c54-947b-ca9b7e215b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:52.179082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-1ed7ac83-5332-4c54-947b-ca9b7e215b2b None None] MFA Rules not processed for user `3561c92204194888a26f9390904ede55`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:52.180351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-1ed7ac83-5332-4c54-947b-ca9b7e215b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:52.248634 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1370/2735] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:35:52.260676 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe84da-b5b5-41a2-8852-a27b107d69ba admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:52.262600 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-d0fe84da-b5b5-41a2-8852-a27b107d69ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:52.327469 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe84da-b5b5-41a2-8852-a27b107d69ba admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:52.331138 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129146f-7d76-473b-aa96-8c44e6689af7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:52.331742 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129146f-7d76-473b-aa96-8c44e6689af7 service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:52.332053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129146f-7d76-473b-aa96-8c44e6689af7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:52.332319 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129146f-7d76-473b-aa96-8c44e6689af7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:52.342416 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129146f-7d76-473b-aa96-8c44e6689af7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:52.343939 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129146f-7d76-473b-aa96-8c44e6689af7 service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:52.353776 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1366/2736] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:35:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:35:55.462688 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7ac83-5332-4c54-947b-ca9b7e215b2b None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:55.482721 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:55.529802 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7ac83-5332-4c54-947b-ca9b7e215b2b None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:55.535619 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c223b0-a3c9-4d5c-ab9f-3e8948169aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:55.536276 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c223b0-a3c9-4d5c-ab9f-3e8948169aa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:55.536550 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c223b0-a3c9-4d5c-ab9f-3e8948169aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:55.536818 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c223b0-a3c9-4d5c-ab9f-3e8948169aa2 admin admin] PATH_INFO: `/v3/users/3561c92204194888a26f9390904ede55` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:55.537946 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c223b0-a3c9-4d5c-ab9f-3e8948169aa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3561c92204194888a26f9390904ede55)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:55.539849 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c223b0-a3c9-4d5c-ab9f-3e8948169aa2 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:55.596853 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-59c223b0-a3c9-4d5c-ab9f-3e8948169aa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3561c92204194888a26f9390904ede55'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:55.597529 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-59c223b0-a3c9-4d5c-ab9f-3e8948169aa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3561c92204194888a26f9390904ede55'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:55.606356 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-59c223b0-a3c9-4d5c-ab9f-3e8948169aa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3561c92204194888a26f9390904ede55'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:55.618997 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-59c223b0-a3c9-4d5c-ab9f-3e8948169aa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3561c92204194888a26f9390904ede55'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:55.625950 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-59c223b0-a3c9-4d5c-ab9f-3e8948169aa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3561c92204194888a26f9390904ede55'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:55.631926 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1371/2737] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:35:55 2022] DELETE /identity/v3/users/3561c92204194888a26f9390904ede55 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:35:55.638798 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9944d-24e0-41c4-b32d-cb197da9af7e None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:55.641296 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9944d-24e0-41c4-b32d-cb197da9af7e None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:55.641296 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9944d-24e0-41c4-b32d-cb197da9af7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:55.661453 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-a0c9944d-24e0-41c4-b32d-cb197da9af7e None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:55.662657 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-a0c9944d-24e0-41c4-b32d-cb197da9af7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:55.731755 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1367/2738] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:35:55.741672 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c223b0-a3c9-4d5c-ab9f-3e8948169aa2 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:55.743139 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-59c223b0-a3c9-4d5c-ab9f-3e8948169aa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:55.803564 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c223b0-a3c9-4d5c-ab9f-3e8948169aa2 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:55.807369 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c7d68-e21c-4237-85cc-db091bcdb7b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:55.808026 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c7d68-e21c-4237-85cc-db091bcdb7b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:55.808298 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c7d68-e21c-4237-85cc-db091bcdb7b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:55.808565 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c7d68-e21c-4237-85cc-db091bcdb7b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:55.817876 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c7d68-e21c-4237-85cc-db091bcdb7b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:55.819236 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c7d68-e21c-4237-85cc-db091bcdb7b3 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:55.829334 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1372/2739] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:35:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:35:56.105029 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9944d-24e0-41c4-b32d-cb197da9af7e None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:56.106523 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-a0c9944d-24e0-41c4-b32d-cb197da9af7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:56.119587 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9944d-24e0-41c4-b32d-cb197da9af7e None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:56.124876 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac121b-ee47-4fb2-8ed9-092cc476212b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:56.125515 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac121b-ee47-4fb2-8ed9-092cc476212b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:56.125947 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac121b-ee47-4fb2-8ed9-092cc476212b admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:56.126223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac121b-ee47-4fb2-8ed9-092cc476212b admin admin] PATH_INFO: `/v3/projects/191c6b14aa0d4277920cd57df4d24c95` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:56.127470 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac121b-ee47-4fb2-8ed9-092cc476212b admin admin] RBAC: Authorizing `identity:delete_project(project_id=191c6b14aa0d4277920cd57df4d24c95)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:56.129423 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac121b-ee47-4fb2-8ed9-092cc476212b admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:56.180709 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9dac121b-ee47-4fb2-8ed9-092cc476212b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '191c6b14aa0d4277920cd57df4d24c95', 'request_id': 'req-9dac121b-ee47-4fb2-8ed9-092cc476212b'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:56.187371 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9dac121b-ee47-4fb2-8ed9-092cc476212b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '191c6b14aa0d4277920cd57df4d24c95', 'request_id': 'req-9dac121b-ee47-4fb2-8ed9-092cc476212b'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:56.195476 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9dac121b-ee47-4fb2-8ed9-092cc476212b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '191c6b14aa0d4277920cd57df4d24c95', 'request_id': 'req-9dac121b-ee47-4fb2-8ed9-092cc476212b'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:56.196881 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:56.208593 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9dac121b-ee47-4fb2-8ed9-092cc476212b admin admin] The token cache is being invalidate because project 191c6b14aa0d4277920cd57df4d24c95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:35:56.208889 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9dac121b-ee47-4fb2-8ed9-092cc476212b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:56.209609 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9dac121b-ee47-4fb2-8ed9-092cc476212b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:56.211241 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1368/2740] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:35:56 2022] DELETE /identity/v3/projects/191c6b14aa0d4277920cd57df4d24c95 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:35:56.218059 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40e156-5ae6-4966-889e-3d1520e4dd17 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:56.218442 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40e156-5ae6-4966-889e-3d1520e4dd17 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:56.218810 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40e156-5ae6-4966-889e-3d1520e4dd17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:56.248171 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-eb40e156-5ae6-4966-889e-3d1520e4dd17 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:56.249309 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-eb40e156-5ae6-4966-889e-3d1520e4dd17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:56.284972 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:56.316221 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1373/2741] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:35:56.324833 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0a5a3-e3df-47f1-833b-f6fcbce1dd83 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:56.325172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0a5a3-e3df-47f1-833b-f6fcbce1dd83 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:56.325456 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0a5a3-e3df-47f1-833b-f6fcbce1dd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:56.345931 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-d1c0a5a3-e3df-47f1-833b-f6fcbce1dd83 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:56.347136 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-d1c0a5a3-e3df-47f1-833b-f6fcbce1dd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:56.356473 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1369/2742] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:35:56.362920 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb40e156-5ae6-4966-889e-3d1520e4dd17 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:56.375330 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb40e156-5ae6-4966-889e-3d1520e4dd17 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:56.381156 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14843531-a630-4790-8203-cebde613d5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:56.381773 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14843531-a630-4790-8203-cebde613d5ed admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:56.382052 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14843531-a630-4790-8203-cebde613d5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:56.382307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14843531-a630-4790-8203-cebde613d5ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:56.382952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14843531-a630-4790-8203-cebde613d5ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:35:56.383256 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14843531-a630-4790-8203-cebde613d5ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:56.384886 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14843531-a630-4790-8203-cebde613d5ed admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:56.398393 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1374/2743] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:35:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:35:56.403240 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0a5a3-e3df-47f1-833b-f6fcbce1dd83 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:56.413682 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0a5a3-e3df-47f1-833b-f6fcbce1dd83 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:56.419694 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d004fee-8fab-4d7c-bbd1-19c88994c28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:56.420393 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d004fee-8fab-4d7c-bbd1-19c88994c28f admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:56.420671 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d004fee-8fab-4d7c-bbd1-19c88994c28f admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:56.420962 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d004fee-8fab-4d7c-bbd1-19c88994c28f admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:56.421554 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d004fee-8fab-4d7c-bbd1-19c88994c28f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:56.423486 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d004fee-8fab-4d7c-bbd1-19c88994c28f admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:56.460337 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1370/2744] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:35:56 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:35:56.461571 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14843531-a630-4790-8203-cebde613d5ed admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:56.472350 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14843531-a630-4790-8203-cebde613d5ed admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:56.529882 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf42ffb-773e-43a3-b17e-314a79833787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:56.530402 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf42ffb-773e-43a3-b17e-314a79833787 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:56.530674 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf42ffb-773e-43a3-b17e-314a79833787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:56.530934 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf42ffb-773e-43a3-b17e-314a79833787 admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:56.531555 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf42ffb-773e-43a3-b17e-314a79833787 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:56.533489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf42ffb-773e-43a3-b17e-314a79833787 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:56.555968 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1375/2745] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:35:56 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:35:56.560773 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d004fee-8fab-4d7c-bbd1-19c88994c28f admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:56.571194 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d004fee-8fab-4d7c-bbd1-19c88994c28f admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:56.576427 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550e6fb-0da8-401c-81c4-3b60aa4e4889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:56.577023 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550e6fb-0da8-401c-81c4-3b60aa4e4889 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:56.577303 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550e6fb-0da8-401c-81c4-3b60aa4e4889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:56.577573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550e6fb-0da8-401c-81c4-3b60aa4e4889 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:56.578218 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8550e6fb-0da8-401c-81c4-3b60aa4e4889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:56.579813 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8550e6fb-0da8-401c-81c4-3b60aa4e4889 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:56.590152 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1371/2746] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:35:56.594204 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf42ffb-773e-43a3-b17e-314a79833787 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:56.608325 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf42ffb-773e-43a3-b17e-314a79833787 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:56.613466 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91535f-4fd5-4f23-adb6-ca81a381e721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:56.614065 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91535f-4fd5-4f23-adb6-ca81a381e721 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:56.614340 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91535f-4fd5-4f23-adb6-ca81a381e721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:56.614633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91535f-4fd5-4f23-adb6-ca81a381e721 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:56.615284 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91535f-4fd5-4f23-adb6-ca81a381e721 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:56.616945 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91535f-4fd5-4f23-adb6-ca81a381e721 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:56.628523 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1376/2747] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:35:56.632835 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550e6fb-0da8-401c-81c4-3b60aa4e4889 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:56.643521 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550e6fb-0da8-401c-81c4-3b60aa4e4889 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:56.649075 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd316e4-be84-4695-b94f-5bf0b996b729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:56.649748 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd316e4-be84-4695-b94f-5bf0b996b729 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:56.650019 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd316e4-be84-4695-b94f-5bf0b996b729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:56.650353 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd316e4-be84-4695-b94f-5bf0b996b729 admin admin] PATH_INFO: `/v3/projects/5964d12c6b8d4c019ffc0e878da6abb9/users/699c6679fee247dd97e9c253bf11716d/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:56.665615 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd316e4-be84-4695-b94f-5bf0b996b729 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5964d12c6b8d4c019ffc0e878da6abb9, user_id=699c6679fee247dd97e9c253bf11716d, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:56.670713 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd316e4-be84-4695-b94f-5bf0b996b729 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:56.689754 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1372/2748] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:35:56 2022] PUT /identity/v3/projects/5964d12c6b8d4c019ffc0e878da6abb9/users/699c6679fee247dd97e9c253bf11716d/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:35:56.758330 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40e724-1c62-4978-9a0f-5493e7579be9 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:56.758608 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40e724-1c62-4978-9a0f-5493e7579be9 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:56.758873 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40e724-1c62-4978-9a0f-5493e7579be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:56.802093 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-5f40e724-1c62-4978-9a0f-5493e7579be9 None None] MFA Rules not processed for user `699c6679fee247dd97e9c253bf11716d`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:56.803736 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-5f40e724-1c62-4978-9a0f-5493e7579be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:56.878513 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1377/2749] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:35:56.883149 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd316e4-be84-4695-b94f-5bf0b996b729 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:56.938739 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f40e724-1c62-4978-9a0f-5493e7579be9 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:56.940270 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-5f40e724-1c62-4978-9a0f-5493e7579be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:56.950180 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd316e4-be84-4695-b94f-5bf0b996b729 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:56.953779 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f40e724-1c62-4978-9a0f-5493e7579be9 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:56.955590 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10899c0c-f5b1-4c16-a041-6f4d95d65aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:56.956279 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10899c0c-f5b1-4c16-a041-6f4d95d65aef admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:56.956553 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10899c0c-f5b1-4c16-a041-6f4d95d65aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:56.956820 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10899c0c-f5b1-4c16-a041-6f4d95d65aef admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:56.957446 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10899c0c-f5b1-4c16-a041-6f4d95d65aef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:56.958517 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead481a-b36a-41c3-9cc3-69dce0d59852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:56.959178 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead481a-b36a-41c3-9cc3-69dce0d59852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:56.959377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10899c0c-f5b1-4c16-a041-6f4d95d65aef admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:56.959479 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead481a-b36a-41c3-9cc3-69dce0d59852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:56.959694 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead481a-b36a-41c3-9cc3-69dce0d59852 admin admin] PATH_INFO: `/v3/users/1a73b8869b374f9d85f16b561a28061b` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:56.960854 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aead481a-b36a-41c3-9cc3-69dce0d59852 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a73b8869b374f9d85f16b561a28061b)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:56.962754 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aead481a-b36a-41c3-9cc3-69dce0d59852 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:56.992549 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1373/2750] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:35:56 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:35:56.993553 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10899c0c-f5b1-4c16-a041-6f4d95d65aef admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:56.998780 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:57.020792 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-aead481a-b36a-41c3-9cc3-69dce0d59852 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a73b8869b374f9d85f16b561a28061b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:57.021476 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-aead481a-b36a-41c3-9cc3-69dce0d59852 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a73b8869b374f9d85f16b561a28061b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:57.029150 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-aead481a-b36a-41c3-9cc3-69dce0d59852 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a73b8869b374f9d85f16b561a28061b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:57.041501 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-aead481a-b36a-41c3-9cc3-69dce0d59852 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a73b8869b374f9d85f16b561a28061b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:57.048463 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-aead481a-b36a-41c3-9cc3-69dce0d59852 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a73b8869b374f9d85f16b561a28061b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:57.054565 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1378/2751] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:35:56 2022] DELETE /identity/v3/users/1a73b8869b374f9d85f16b561a28061b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:35:57.058725 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10899c0c-f5b1-4c16-a041-6f4d95d65aef admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:57.058997 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0565ce-4a54-4574-ad07-82acd405c7d2 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:57.059346 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0565ce-4a54-4574-ad07-82acd405c7d2 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:57.059616 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0565ce-4a54-4574-ad07-82acd405c7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:57.072755 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:57.078157 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-8b0565ce-4a54-4574-ad07-82acd405c7d2 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:57.079422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-8b0565ce-4a54-4574-ad07-82acd405c7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:57.121872 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697eac08-3bdd-415e-930b-e5c39a8632f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:57.122619 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697eac08-3bdd-415e-930b-e5c39a8632f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:57.123052 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697eac08-3bdd-415e-930b-e5c39a8632f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:57.123336 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697eac08-3bdd-415e-930b-e5c39a8632f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:57.123967 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697eac08-3bdd-415e-930b-e5c39a8632f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:57.125885 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697eac08-3bdd-415e-930b-e5c39a8632f5 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:57.138595 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1379/2752] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:35:57.148608 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0565ce-4a54-4574-ad07-82acd405c7d2 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:57.149786 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1374/2753] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:35:56 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:35:57.149955 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-8b0565ce-4a54-4574-ad07-82acd405c7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:57.154305 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697eac08-3bdd-415e-930b-e5c39a8632f5 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:57.156443 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-697eac08-3bdd-415e-930b-e5c39a8632f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:57.168553 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697eac08-3bdd-415e-930b-e5c39a8632f5 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:57.173147 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ad70e-bfbf-41e5-bb7e-69455f3d788b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:57.173754 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ad70e-bfbf-41e5-bb7e-69455f3d788b admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:57.174036 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ad70e-bfbf-41e5-bb7e-69455f3d788b admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:57.174297 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ad70e-bfbf-41e5-bb7e-69455f3d788b admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:57.174940 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ad70e-bfbf-41e5-bb7e-69455f3d788b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:57.176585 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ad70e-bfbf-41e5-bb7e-69455f3d788b admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:57.186928 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1375/2754] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:35:57.190798 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ad70e-bfbf-41e5-bb7e-69455f3d788b admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:57.200712 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ad70e-bfbf-41e5-bb7e-69455f3d788b admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:57.205439 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b46a06-2bff-4783-bf80-b78b03faed33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:57.206031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b46a06-2bff-4783-bf80-b78b03faed33 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:57.206306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b46a06-2bff-4783-bf80-b78b03faed33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:57.206577 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b46a06-2bff-4783-bf80-b78b03faed33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:57.207202 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b46a06-2bff-4783-bf80-b78b03faed33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:57.208903 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b46a06-2bff-4783-bf80-b78b03faed33 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:57.209524 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0565ce-4a54-4574-ad07-82acd405c7d2 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:57.213626 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6058fcc-f3ea-4fb2-8e06-94c3e90ad1c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:57.214233 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6058fcc-f3ea-4fb2-8e06-94c3e90ad1c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:57.214512 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6058fcc-f3ea-4fb2-8e06-94c3e90ad1c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:57.214782 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6058fcc-f3ea-4fb2-8e06-94c3e90ad1c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:57.219437 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1376/2755] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:35:57.223479 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b46a06-2bff-4783-bf80-b78b03faed33 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:57.225006 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6058fcc-f3ea-4fb2-8e06-94c3e90ad1c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:57.226793 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6058fcc-f3ea-4fb2-8e06-94c3e90ad1c7 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:57.233507 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b46a06-2bff-4783-bf80-b78b03faed33 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:57.237948 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1380/2756] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:35:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:35:57.238365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da33664-d870-4d4d-bd78-61db20e6a0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:57.239028 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da33664-d870-4d4d-bd78-61db20e6a0a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:57.239313 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da33664-d870-4d4d-bd78-61db20e6a0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:57.239584 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da33664-d870-4d4d-bd78-61db20e6a0a4 admin admin] PATH_INFO: `/v3/projects/653fe1fcf851402b95ab79bc8dd2d0de/users/ffa44410b8d64aacb93ad5604cf3d5f8/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:57.254533 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da33664-d870-4d4d-bd78-61db20e6a0a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=653fe1fcf851402b95ab79bc8dd2d0de, user_id=ffa44410b8d64aacb93ad5604cf3d5f8, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:57.257315 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da33664-d870-4d4d-bd78-61db20e6a0a4 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:57.271581 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1377/2757] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:35:57 2022] PUT /identity/v3/projects/653fe1fcf851402b95ab79bc8dd2d0de/users/ffa44410b8d64aacb93ad5604cf3d5f8/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:35:57.282046 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a94ffc-b121-417c-b0d2-afd659d79462 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:57.282334 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a94ffc-b121-417c-b0d2-afd659d79462 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:57.282611 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a94ffc-b121-417c-b0d2-afd659d79462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:57.311573 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-40a94ffc-b121-417c-b0d2-afd659d79462 None None] MFA Rules not processed for user `ffa44410b8d64aacb93ad5604cf3d5f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:57.312733 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-40a94ffc-b121-417c-b0d2-afd659d79462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:57.351784 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:57.382189 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1381/2758] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:35:57.387149 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da33664-d870-4d4d-bd78-61db20e6a0a4 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:57.458377 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da33664-d870-4d4d-bd78-61db20e6a0a4 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:57.463157 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c028a49-7bab-40fa-b324-e00810197693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:57.463732 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c028a49-7bab-40fa-b324-e00810197693 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:57.464045 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c028a49-7bab-40fa-b324-e00810197693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:57.464325 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c028a49-7bab-40fa-b324-e00810197693 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:57.464972 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c028a49-7bab-40fa-b324-e00810197693 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:57.466949 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c028a49-7bab-40fa-b324-e00810197693 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:57.496159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1378/2759] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:35:57 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:35:57.501364 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a94ffc-b121-417c-b0d2-afd659d79462 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:57.512429 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a94ffc-b121-417c-b0d2-afd659d79462 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:57.522139 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c028a49-7bab-40fa-b324-e00810197693 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:57.523897 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-3c028a49-7bab-40fa-b324-e00810197693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:57.577185 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:57.585035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c56f26-0558-4379-bf7d-3ac2ee30022b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:57.585657 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c56f26-0558-4379-bf7d-3ac2ee30022b admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:57.585938 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c56f26-0558-4379-bf7d-3ac2ee30022b admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:57.586207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c56f26-0558-4379-bf7d-3ac2ee30022b admin admin] PATH_INFO: `/v3/users` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:57.586848 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c56f26-0558-4379-bf7d-3ac2ee30022b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:57.588987 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c56f26-0558-4379-bf7d-3ac2ee30022b admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:57.594015 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c028a49-7bab-40fa-b324-e00810197693 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:57.598870 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ed2c0-d07f-4364-870d-bd2b6c1a3376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:57.599512 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ed2c0-d07f-4364-870d-bd2b6c1a3376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:57.599788 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ed2c0-d07f-4364-870d-bd2b6c1a3376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:57.600120 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ed2c0-d07f-4364-870d-bd2b6c1a3376 admin admin] PATH_INFO: `/v3/projects/de57031919624c4d8f5e3a2ddcb90bfd` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:57.601594 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ed2c0-d07f-4364-870d-bd2b6c1a3376 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de57031919624c4d8f5e3a2ddcb90bfd)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:57.603933 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ed2c0-d07f-4364-870d-bd2b6c1a3376 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:57.611517 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1382/2760] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:35:57 2022] POST /identity/v3/users => generated 298 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:35:57.615686 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c56f26-0558-4379-bf7d-3ac2ee30022b admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:57.626170 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c56f26-0558-4379-bf7d-3ac2ee30022b admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:57.630779 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d723cd-7360-479c-9c4c-baec3d341703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:57.631370 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d723cd-7360-479c-9c4c-baec3d341703 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:57.631653 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d723cd-7360-479c-9c4c-baec3d341703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:57.631935 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d723cd-7360-479c-9c4c-baec3d341703 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:57.632623 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d723cd-7360-479c-9c4c-baec3d341703 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:57.634745 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d723cd-7360-479c-9c4c-baec3d341703 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:57.646173 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1383/2761] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:35:57.651907 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d723cd-7360-479c-9c4c-baec3d341703 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:57.660359 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-611ed2c0-d07f-4364-870d-bd2b6c1a3376 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de57031919624c4d8f5e3a2ddcb90bfd', 'request_id': 'req-611ed2c0-d07f-4364-870d-bd2b6c1a3376'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:57.661475 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d723cd-7360-479c-9c4c-baec3d341703 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:57.667248 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-611ed2c0-d07f-4364-870d-bd2b6c1a3376 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de57031919624c4d8f5e3a2ddcb90bfd', 'request_id': 'req-611ed2c0-d07f-4364-870d-bd2b6c1a3376'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:57.675509 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-611ed2c0-d07f-4364-870d-bd2b6c1a3376 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de57031919624c4d8f5e3a2ddcb90bfd', 'request_id': 'req-611ed2c0-d07f-4364-870d-bd2b6c1a3376'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:57.693238 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-611ed2c0-d07f-4364-870d-bd2b6c1a3376 admin admin] The token cache is being invalidate because project de57031919624c4d8f5e3a2ddcb90bfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:35:57.693876 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-611ed2c0-d07f-4364-870d-bd2b6c1a3376 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:57.695187 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-611ed2c0-d07f-4364-870d-bd2b6c1a3376 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:35:57.699802 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1379/2762] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:35:57 2022] DELETE /identity/v3/projects/de57031919624c4d8f5e3a2ddcb90bfd => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:35:57.705251 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44dee3-6821-4958-b620-ad27426faf0b None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:57.705540 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44dee3-6821-4958-b620-ad27426faf0b None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:57.705848 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44dee3-6821-4958-b620-ad27426faf0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:57.722211 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8e0a5-01ca-4ade-8e4a-5bcf29313a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:57.722878 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8e0a5-01ca-4ade-8e4a-5bcf29313a21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:57.723156 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8e0a5-01ca-4ade-8e4a-5bcf29313a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:57.723466 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8e0a5-01ca-4ade-8e4a-5bcf29313a21 admin admin] PATH_INFO: `/v3/projects/9b8bce2486aa400c9b279b9343758dc9/users/3252192f00384eceb044566e60b3533b/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:57.729605 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-6b44dee3-6821-4958-b620-ad27426faf0b None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:57.731676 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-6b44dee3-6821-4958-b620-ad27426faf0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:57.740019 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8e0a5-01ca-4ade-8e4a-5bcf29313a21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b8bce2486aa400c9b279b9343758dc9, user_id=3252192f00384eceb044566e60b3533b, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:57.741132 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1380/2763] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:35:57.742634 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8e0a5-01ca-4ade-8e4a-5bcf29313a21 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:57.749141 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122bd0c-f562-49b3-9890-bcdf699764fe None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:57.749500 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122bd0c-f562-49b3-9890-bcdf699764fe None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:57.749780 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122bd0c-f562-49b3-9890-bcdf699764fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:57.762856 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1384/2764] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:35:57 2022] PUT /identity/v3/projects/9b8bce2486aa400c9b279b9343758dc9/users/3252192f00384eceb044566e60b3533b/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:35:57.766795 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8e0a5-01ca-4ade-8e4a-5bcf29313a21 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:57.768411 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-84e8e0a5-01ca-4ade-8e4a-5bcf29313a21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:57.771645 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-2122bd0c-f562-49b3-9890-bcdf699764fe None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:57.772904 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-2122bd0c-f562-49b3-9890-bcdf699764fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:57.810911 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:57.832984 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1381/2765] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:35:57.833908 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8e0a5-01ca-4ade-8e4a-5bcf29313a21 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:57.837347 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2122bd0c-f562-49b3-9890-bcdf699764fe None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:57.838680 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d23c38-1c6f-41a3-88eb-7dd4da2a60ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:57.839279 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d23c38-1c6f-41a3-88eb-7dd4da2a60ec admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:57.839555 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d23c38-1c6f-41a3-88eb-7dd4da2a60ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:57.839823 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d23c38-1c6f-41a3-88eb-7dd4da2a60ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:57.840498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d23c38-1c6f-41a3-88eb-7dd4da2a60ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:57.842082 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d23c38-1c6f-41a3-88eb-7dd4da2a60ec admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:57.846639 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2122bd0c-f562-49b3-9890-bcdf699764fe None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:57.851838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9c70c-2861-4456-915c-dcf08b28239a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:57.852359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1385/2766] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:35:57.852497 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9c70c-2861-4456-915c-dcf08b28239a admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:57.852738 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9c70c-2861-4456-915c-dcf08b28239a admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:57.853008 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9c70c-2861-4456-915c-dcf08b28239a admin admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:57.853689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b9c70c-2861-4456-915c-dcf08b28239a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:35:57.853991 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b9c70c-2861-4456-915c-dcf08b28239a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:57.855592 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b9c70c-2861-4456-915c-dcf08b28239a admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:57.856569 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d23c38-1c6f-41a3-88eb-7dd4da2a60ec admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:57.866362 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d23c38-1c6f-41a3-88eb-7dd4da2a60ec admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:57.867503 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1382/2767] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 04:35:57.871370 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9accbd-a810-487b-8324-319c3a0488d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:57.872033 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9c70c-2861-4456-915c-dcf08b28239a admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:57.872168 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9accbd-a810-487b-8324-319c3a0488d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:57.872354 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9accbd-a810-487b-8324-319c3a0488d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:57.872619 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9accbd-a810-487b-8324-319c3a0488d7 admin admin] PATH_INFO: `/v3/projects/9b8bce2486aa400c9b279b9343758dc9/users/3252192f00384eceb044566e60b3533b/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:57.874567 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9accbd-a810-487b-8324-319c3a0488d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b8bce2486aa400c9b279b9343758dc9, user_id=3252192f00384eceb044566e60b3533b, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:57.877251 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9accbd-a810-487b-8324-319c3a0488d7 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:57.882473 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9c70c-2861-4456-915c-dcf08b28239a admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:57.887837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1386/2768] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:35:57 2022] PUT /identity/v3/projects/9b8bce2486aa400c9b279b9343758dc9/users/3252192f00384eceb044566e60b3533b/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:35:57.897858 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447d85e-2e79-4f9c-b533-e1e2b4135d69 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:57.898148 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447d85e-2e79-4f9c-b533-e1e2b4135d69 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:57.898423 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447d85e-2e79-4f9c-b533-e1e2b4135d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:57.925798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-8447d85e-2e79-4f9c-b533-e1e2b4135d69 None None] MFA Rules not processed for user `3252192f00384eceb044566e60b3533b`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:57.927153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-8447d85e-2e79-4f9c-b533-e1e2b4135d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:57.934384 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:57.941206 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b7d5b-4719-47ed-949b-e68a362eb0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:57.941827 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b7d5b-4719-47ed-949b-e68a362eb0ad admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:57.942350 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b7d5b-4719-47ed-949b-e68a362eb0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:57.942698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b7d5b-4719-47ed-949b-e68a362eb0ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:57.943524 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b7d5b-4719-47ed-949b-e68a362eb0ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:57.945965 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b7d5b-4719-47ed-949b-e68a362eb0ad admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:57.975573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1383/2769] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:35:57 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:35:57.979984 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b7d5b-4719-47ed-949b-e68a362eb0ad admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:58.002099 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1387/2770] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:57 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:35:58.015055 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447d85e-2e79-4f9c-b533-e1e2b4135d69 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:58.016571 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-8447d85e-2e79-4f9c-b533-e1e2b4135d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:58.049614 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b7d5b-4719-47ed-949b-e68a362eb0ad admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:58.054417 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118bfbe-b2c9-416f-a54b-f3489626cb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:58.055030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118bfbe-b2c9-416f-a54b-f3489626cb8d admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:58.055304 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118bfbe-b2c9-416f-a54b-f3489626cb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:58.055572 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118bfbe-b2c9-416f-a54b-f3489626cb8d admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:58.056225 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b118bfbe-b2c9-416f-a54b-f3489626cb8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:58.058458 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b118bfbe-b2c9-416f-a54b-f3489626cb8d admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:58.078487 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447d85e-2e79-4f9c-b533-e1e2b4135d69 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:58.085779 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1384/2771] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:35:57 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:35:58.089813 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118bfbe-b2c9-416f-a54b-f3489626cb8d admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:58.100281 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118bfbe-b2c9-416f-a54b-f3489626cb8d admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:58.105863 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1280d859-543f-4ca0-853c-93d45f1aeb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:58.106466 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280d859-543f-4ca0-853c-93d45f1aeb20 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:58.106739 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280d859-543f-4ca0-853c-93d45f1aeb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:58.107005 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280d859-543f-4ca0-853c-93d45f1aeb20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:58.107642 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1280d859-543f-4ca0-853c-93d45f1aeb20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:58.109255 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1280d859-543f-4ca0-853c-93d45f1aeb20 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:58.122521 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1385/2772] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:35:58.123569 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1280d859-543f-4ca0-853c-93d45f1aeb20 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:58.133298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1280d859-543f-4ca0-853c-93d45f1aeb20 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:58.133962 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57d16b-3539-426a-968f-e5dd98d48b10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:58.134615 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57d16b-3539-426a-968f-e5dd98d48b10 service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:58.134919 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57d16b-3539-426a-968f-e5dd98d48b10 service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:58.135188 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57d16b-3539-426a-968f-e5dd98d48b10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:58.136927 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-4f57d16b-3539-426a-968f-e5dd98d48b10 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:58.138605 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4391b-5e0a-4079-aa2e-710ba60f6533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:58.139199 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4391b-5e0a-4079-aa2e-710ba60f6533 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:58.139480 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4391b-5e0a-4079-aa2e-710ba60f6533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:58.139752 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4391b-5e0a-4079-aa2e-710ba60f6533 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:58.140454 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f4391b-5e0a-4079-aa2e-710ba60f6533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:58.142030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f4391b-5e0a-4079-aa2e-710ba60f6533 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:58.153070 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1386/2773] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:35:58.155306 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:58.157487 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4391b-5e0a-4079-aa2e-710ba60f6533 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:58.167800 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4391b-5e0a-4079-aa2e-710ba60f6533 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:58.172770 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a19577-afcc-4781-ae6f-57a84dd1528a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:58.173409 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a19577-afcc-4781-ae6f-57a84dd1528a admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:58.173689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a19577-afcc-4781-ae6f-57a84dd1528a admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:58.173960 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a19577-afcc-4781-ae6f-57a84dd1528a admin admin] PATH_INFO: `/v3/projects/0fa7aa0063804717a00a59ea3e188779/users/6a7ac4048d0646f78c0d813fa0112a83/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:58.189014 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a19577-afcc-4781-ae6f-57a84dd1528a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fa7aa0063804717a00a59ea3e188779, user_id=6a7ac4048d0646f78c0d813fa0112a83, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:58.191792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a19577-afcc-4781-ae6f-57a84dd1528a admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:58.199667 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f57d16b-3539-426a-968f-e5dd98d48b10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:58.201290 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f57d16b-3539-426a-968f-e5dd98d48b10 service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:58.206307 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1387/2774] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:35:58 2022] PUT /identity/v3/projects/0fa7aa0063804717a00a59ea3e188779/users/6a7ac4048d0646f78c0d813fa0112a83/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:35:58.210961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1388/2775] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:35:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:35:58.211215 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602eeec9-8161-4553-a6fa-fb9b62bcea81 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:58.211504 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602eeec9-8161-4553-a6fa-fb9b62bcea81 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:58.211785 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602eeec9-8161-4553-a6fa-fb9b62bcea81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:58.231774 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-602eeec9-8161-4553-a6fa-fb9b62bcea81 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:35:58.233026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-602eeec9-8161-4553-a6fa-fb9b62bcea81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:58.295129 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1388/2776] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:35:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:35:58.304962 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57d16b-3539-426a-968f-e5dd98d48b10 service cinder] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:58.306523 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-4f57d16b-3539-426a-968f-e5dd98d48b10 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:58.370279 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57d16b-3539-426a-968f-e5dd98d48b10 service cinder] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:58.421897 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59886b52-5b7e-499d-943a-9d8fa2af9238 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:58.422550 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59886b52-5b7e-499d-943a-9d8fa2af9238 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:58.422853 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59886b52-5b7e-499d-943a-9d8fa2af9238 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:58.423157 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59886b52-5b7e-499d-943a-9d8fa2af9238 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:58.432538 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59886b52-5b7e-499d-943a-9d8fa2af9238 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:58.434076 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59886b52-5b7e-499d-943a-9d8fa2af9238 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:58.443836 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1389/2777] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:35:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:35:59.751305 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602eeec9-8161-4553-a6fa-fb9b62bcea81 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:35:59.753229 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-602eeec9-8161-4553-a6fa-fb9b62bcea81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:59.788499 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:35:59.826976 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602eeec9-8161-4553-a6fa-fb9b62bcea81 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:35:59.831053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48053e-92c2-459e-88f1-ca31804c840d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:35:59.831671 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48053e-92c2-459e-88f1-ca31804c840d service cinder] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:35:59.831975 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48053e-92c2-459e-88f1-ca31804c840d service cinder] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:35:59.832249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48053e-92c2-459e-88f1-ca31804c840d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:35:59.834156 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-cc48053e-92c2-459e-88f1-ca31804c840d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:35:59.920729 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc48053e-92c2-459e-88f1-ca31804c840d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:35:59.922277 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc48053e-92c2-459e-88f1-ca31804c840d service cinder] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:35:59.933795 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1389/2778] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:35:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:36:00.549815 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59886b52-5b7e-499d-943a-9d8fa2af9238 service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:36:00.560246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59886b52-5b7e-499d-943a-9d8fa2af9238 service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:36:00.566179 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5a996-03c0-48a6-b22f-485e40f8d36a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:36:00.566954 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5a996-03c0-48a6-b22f-485e40f8d36a service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:00.567233 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5a996-03c0-48a6-b22f-485e40f8d36a service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:00.567671 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5a996-03c0-48a6-b22f-485e40f8d36a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:00.698186 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5a996-03c0-48a6-b22f-485e40f8d36a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:36:00.700249 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5a996-03c0-48a6-b22f-485e40f8d36a service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:36:00.712340 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1390/2779] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:36:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:36:02.413716 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb521d-6b66-42a3-a3d1-fb79b2d5a28f None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:02.414157 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb521d-6b66-42a3-a3d1-fb79b2d5a28f None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:02.414157 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb521d-6b66-42a3-a3d1-fb79b2d5a28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:02.449808 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-99fb521d-6b66-42a3-a3d1-fb79b2d5a28f None None] MFA Rules not processed for user `6a7ac4048d0646f78c0d813fa0112a83`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:36:02.451165 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-99fb521d-6b66-42a3-a3d1-fb79b2d5a28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:36:02.535752 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1390/2780] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:36:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:36:02.540246 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5a996-03c0-48a6-b22f-485e40f8d36a service cinder] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:36:02.552180 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5a996-03c0-48a6-b22f-485e40f8d36a service cinder] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:36:02.558175 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cd9c23-1946-4575-98ee-4ba7890e3088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:36:02.558891 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd9c23-1946-4575-98ee-4ba7890e3088 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:02.559385 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd9c23-1946-4575-98ee-4ba7890e3088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:02.559895 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd9c23-1946-4575-98ee-4ba7890e3088 admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:02.560673 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cd9c23-1946-4575-98ee-4ba7890e3088 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:36:02.562711 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cd9c23-1946-4575-98ee-4ba7890e3088 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:36:02.574088 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=100 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:36:02.595812 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1391/2781] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:36:02 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:36:02.599697 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb521d-6b66-42a3-a3d1-fb79b2d5a28f None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:36:02.616984 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb521d-6b66-42a3-a3d1-fb79b2d5a28f None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:36:02.680599 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca9005-3cbc-4213-a750-16aef6299444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:36:02.681429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca9005-3cbc-4213-a750-16aef6299444 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:02.681910 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca9005-3cbc-4213-a750-16aef6299444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:02.682527 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca9005-3cbc-4213-a750-16aef6299444 admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:02.682792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ca9005-3cbc-4213-a750-16aef6299444 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:36:02.684809 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ca9005-3cbc-4213-a750-16aef6299444 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:36:02.703945 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:36:02.714391 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1391/2782] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:36:02 2022] POST /identity/v3/users => generated 292 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:36:02.715520 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cd9c23-1946-4575-98ee-4ba7890e3088 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:36:02.727712 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cd9c23-1946-4575-98ee-4ba7890e3088 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:36:02.733294 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494b30f-5497-4c74-9bd6-7f6b1ce143e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:36:02.733695 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494b30f-5497-4c74-9bd6-7f6b1ce143e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:02.734262 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494b30f-5497-4c74-9bd6-7f6b1ce143e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:02.734604 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494b30f-5497-4c74-9bd6-7f6b1ce143e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:02.735331 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5494b30f-5497-4c74-9bd6-7f6b1ce143e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:36:02.737066 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5494b30f-5497-4c74-9bd6-7f6b1ce143e8 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:36:02.748652 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1392/2783] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:36:02.753336 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca9005-3cbc-4213-a750-16aef6299444 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:36:02.766596 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca9005-3cbc-4213-a750-16aef6299444 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:36:02.771815 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0c41d-88f1-48eb-b48b-abbdd9dbc647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:36:02.772540 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0c41d-88f1-48eb-b48b-abbdd9dbc647 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:02.772821 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0c41d-88f1-48eb-b48b-abbdd9dbc647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:02.773094 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0c41d-88f1-48eb-b48b-abbdd9dbc647 admin admin] PATH_INFO: `/v3/projects/e283a55c05814e32a0cbd231bfa2a4e0/users/f15ce0909d8848f48c53e4e3868e9cef/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:02.794817 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a0c41d-88f1-48eb-b48b-abbdd9dbc647 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e283a55c05814e32a0cbd231bfa2a4e0, user_id=f15ce0909d8848f48c53e4e3868e9cef, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:36:02.798220 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a0c41d-88f1-48eb-b48b-abbdd9dbc647 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:36:02.816792 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1392/2784] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:36:02 2022] PUT /identity/v3/projects/e283a55c05814e32a0cbd231bfa2a4e0/users/f15ce0909d8848f48c53e4e3868e9cef/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:36:02.820986 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494b30f-5497-4c74-9bd6-7f6b1ce143e8 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:36:02.832593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494b30f-5497-4c74-9bd6-7f6b1ce143e8 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:36:02.900976 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5150e7ee-6166-4b9a-9e45-81ba753909c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:36:02.901708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150e7ee-6166-4b9a-9e45-81ba753909c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:02.902076 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150e7ee-6166-4b9a-9e45-81ba753909c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:02.902456 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150e7ee-6166-4b9a-9e45-81ba753909c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:02.903189 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5150e7ee-6166-4b9a-9e45-81ba753909c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:36:02.904832 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5150e7ee-6166-4b9a-9e45-81ba753909c8 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:36:02.915602 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1393/2785] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:36:02.920275 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0c41d-88f1-48eb-b48b-abbdd9dbc647 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:36:02.930482 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0c41d-88f1-48eb-b48b-abbdd9dbc647 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:36:02.935363 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b076295-4e44-4b7b-b322-f522d16b5eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:36:02.936087 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b076295-4e44-4b7b-b322-f522d16b5eaf admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:02.936365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b076295-4e44-4b7b-b322-f522d16b5eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:02.936632 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b076295-4e44-4b7b-b322-f522d16b5eaf admin admin] PATH_INFO: `/v3/projects/e283a55c05814e32a0cbd231bfa2a4e0/users/f15ce0909d8848f48c53e4e3868e9cef/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:02.938545 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b076295-4e44-4b7b-b322-f522d16b5eaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e283a55c05814e32a0cbd231bfa2a4e0, user_id=f15ce0909d8848f48c53e4e3868e9cef, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:36:02.941224 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b076295-4e44-4b7b-b322-f522d16b5eaf admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:36:02.954264 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1393/2786] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:36:02 2022] PUT /identity/v3/projects/e283a55c05814e32a0cbd231bfa2a4e0/users/f15ce0909d8848f48c53e4e3868e9cef/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:36:04.819017 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5150e7ee-6166-4b9a-9e45-81ba753909c8 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:36:04.835015 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5150e7ee-6166-4b9a-9e45-81ba753909c8 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:36:04.908947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746664c-c032-4a75-8a8d-f71167dbbae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:36:04.909599 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746664c-c032-4a75-8a8d-f71167dbbae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:04.909872 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746664c-c032-4a75-8a8d-f71167dbbae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:04.910143 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746664c-c032-4a75-8a8d-f71167dbbae1 admin admin] PATH_INFO: `/v3/users/699c6679fee247dd97e9c253bf11716d` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:04.911278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c746664c-c032-4a75-8a8d-f71167dbbae1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=699c6679fee247dd97e9c253bf11716d)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:36:04.913545 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c746664c-c032-4a75-8a8d-f71167dbbae1 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:36:04.978968 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c746664c-c032-4a75-8a8d-f71167dbbae1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '699c6679fee247dd97e9c253bf11716d'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:36:04.980719 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c746664c-c032-4a75-8a8d-f71167dbbae1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '699c6679fee247dd97e9c253bf11716d'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:36:04.992842 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c746664c-c032-4a75-8a8d-f71167dbbae1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '699c6679fee247dd97e9c253bf11716d'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:36:05.011572 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c746664c-c032-4a75-8a8d-f71167dbbae1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '699c6679fee247dd97e9c253bf11716d'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:36:05.016433 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:36:05.019767 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-c746664c-c032-4a75-8a8d-f71167dbbae1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '699c6679fee247dd97e9c253bf11716d'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:36:05.030461 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1394/2787] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:36:04 2022] DELETE /identity/v3/users/699c6679fee247dd97e9c253bf11716d => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:36:05.033385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8fdfa-037b-4a08-a275-c3f5337dea83 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:05.033929 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8fdfa-037b-4a08-a275-c3f5337dea83 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:05.033929 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8fdfa-037b-4a08-a275-c3f5337dea83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:05.060408 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-f3c8fdfa-037b-4a08-a275-c3f5337dea83 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:36:05.061626 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-f3c8fdfa-037b-4a08-a275-c3f5337dea83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:36:05.180589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1394/2788] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:36:05.190675 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746664c-c032-4a75-8a8d-f71167dbbae1 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:36:05.193133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-c746664c-c032-4a75-8a8d-f71167dbbae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:36:05.283091 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746664c-c032-4a75-8a8d-f71167dbbae1 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:36:05.287153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e3390-aa8c-4cf6-b011-a9749b70e866 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:36:05.287885 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e3390-aa8c-4cf6-b011-a9749b70e866 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:05.288287 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e3390-aa8c-4cf6-b011-a9749b70e866 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:05.288619 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e3390-aa8c-4cf6-b011-a9749b70e866 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:05.300599 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e3390-aa8c-4cf6-b011-a9749b70e866 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:36:05.302590 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e3390-aa8c-4cf6-b011-a9749b70e866 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:36:05.325019 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1395/2789] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:36:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:36:05.724913 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c8fdfa-037b-4a08-a275-c3f5337dea83 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:36:05.726998 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-f3c8fdfa-037b-4a08-a275-c3f5337dea83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:36:05.754296 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c8fdfa-037b-4a08-a275-c3f5337dea83 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:36:05.772347 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bda39e-a9cb-4884-9d7b-25c153fa7e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:36:05.773017 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bda39e-a9cb-4884-9d7b-25c153fa7e8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:05.773293 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bda39e-a9cb-4884-9d7b-25c153fa7e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:05.773620 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bda39e-a9cb-4884-9d7b-25c153fa7e8c admin admin] PATH_INFO: `/v3/projects/5964d12c6b8d4c019ffc0e878da6abb9` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:05.775008 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bda39e-a9cb-4884-9d7b-25c153fa7e8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5964d12c6b8d4c019ffc0e878da6abb9)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:36:05.778413 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bda39e-a9cb-4884-9d7b-25c153fa7e8c admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:36:05.888762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-39bda39e-a9cb-4884-9d7b-25c153fa7e8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5964d12c6b8d4c019ffc0e878da6abb9', 'request_id': 'req-39bda39e-a9cb-4884-9d7b-25c153fa7e8c'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:36:05.901366 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-39bda39e-a9cb-4884-9d7b-25c153fa7e8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5964d12c6b8d4c019ffc0e878da6abb9', 'request_id': 'req-39bda39e-a9cb-4884-9d7b-25c153fa7e8c'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:36:05.918562 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-39bda39e-a9cb-4884-9d7b-25c153fa7e8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5964d12c6b8d4c019ffc0e878da6abb9', 'request_id': 'req-39bda39e-a9cb-4884-9d7b-25c153fa7e8c'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:36:05.941237 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-39bda39e-a9cb-4884-9d7b-25c153fa7e8c admin admin] The token cache is being invalidate because project 5964d12c6b8d4c019ffc0e878da6abb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:36:05.944203 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-39bda39e-a9cb-4884-9d7b-25c153fa7e8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:36:05.948109 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-39bda39e-a9cb-4884-9d7b-25c153fa7e8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:36:05.950681 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1395/2790] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:36:05 2022] DELETE /identity/v3/projects/5964d12c6b8d4c019ffc0e878da6abb9 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:36:05.954525 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e3390-aa8c-4cf6-b011-a9749b70e866 service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:36:05.976597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e3390-aa8c-4cf6-b011-a9749b70e866 service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:36:06.106535 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748a9b4-e8c2-4def-8049-98409971c56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:36:06.107285 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748a9b4-e8c2-4def-8049-98409971c56a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:06.107631 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748a9b4-e8c2-4def-8049-98409971c56a admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:06.108020 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748a9b4-e8c2-4def-8049-98409971c56a admin admin] PATH_INFO: `/v3/users/ffa44410b8d64aacb93ad5604cf3d5f8` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:06.111504 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6748a9b4-e8c2-4def-8049-98409971c56a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffa44410b8d64aacb93ad5604cf3d5f8)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:36:06.114730 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6748a9b4-e8c2-4def-8049-98409971c56a admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:36:06.204263 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-6748a9b4-e8c2-4def-8049-98409971c56a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffa44410b8d64aacb93ad5604cf3d5f8'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:36:06.205151 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-6748a9b4-e8c2-4def-8049-98409971c56a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffa44410b8d64aacb93ad5604cf3d5f8'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:36:06.218808 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-6748a9b4-e8c2-4def-8049-98409971c56a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffa44410b8d64aacb93ad5604cf3d5f8'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:36:06.249314 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-6748a9b4-e8c2-4def-8049-98409971c56a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffa44410b8d64aacb93ad5604cf3d5f8'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:36:06.261546 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-6748a9b4-e8c2-4def-8049-98409971c56a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffa44410b8d64aacb93ad5604cf3d5f8'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:36:06.271606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1396/2791] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:36:05 2022] DELETE /identity/v3/users/ffa44410b8d64aacb93ad5604cf3d5f8 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:36:06.802689 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bda39e-a9cb-4884-9d7b-25c153fa7e8c admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:36:06.804256 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-39bda39e-a9cb-4884-9d7b-25c153fa7e8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:36:06.833857 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bda39e-a9cb-4884-9d7b-25c153fa7e8c admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:36:06.908517 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:36:06.926347 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9505561e-2c64-42a8-836a-bc0f594638c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:36:06.927064 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505561e-2c64-42a8-836a-bc0f594638c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:06.927300 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505561e-2c64-42a8-836a-bc0f594638c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:06.929565 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505561e-2c64-42a8-836a-bc0f594638c8 admin admin] PATH_INFO: `/v3/projects/653fe1fcf851402b95ab79bc8dd2d0de` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:06.931419 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9505561e-2c64-42a8-836a-bc0f594638c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=653fe1fcf851402b95ab79bc8dd2d0de)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:36:06.933939 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9505561e-2c64-42a8-836a-bc0f594638c8 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:36:07.032262 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9505561e-2c64-42a8-836a-bc0f594638c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '653fe1fcf851402b95ab79bc8dd2d0de', 'request_id': 'req-9505561e-2c64-42a8-836a-bc0f594638c8'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:36:07.066659 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9505561e-2c64-42a8-836a-bc0f594638c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '653fe1fcf851402b95ab79bc8dd2d0de', 'request_id': 'req-9505561e-2c64-42a8-836a-bc0f594638c8'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:36:07.079601 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9505561e-2c64-42a8-836a-bc0f594638c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '653fe1fcf851402b95ab79bc8dd2d0de', 'request_id': 'req-9505561e-2c64-42a8-836a-bc0f594638c8'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:36:07.094432 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9505561e-2c64-42a8-836a-bc0f594638c8 admin admin] The token cache is being invalidate because project 653fe1fcf851402b95ab79bc8dd2d0de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:36:07.094735 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9505561e-2c64-42a8-836a-bc0f594638c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:36:07.095717 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-9505561e-2c64-42a8-836a-bc0f594638c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:36:07.108221 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1396/2792] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:36:06 2022] DELETE /identity/v3/projects/653fe1fcf851402b95ab79bc8dd2d0de => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:36:07.109556 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748a9b4-e8c2-4def-8049-98409971c56a admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:36:07.111022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-6748a9b4-e8c2-4def-8049-98409971c56a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:36:07.138178 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748a9b4-e8c2-4def-8049-98409971c56a admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:36:07.158249 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:36:07.230156 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d85713-a785-43d2-8bf1-ac18f65b0531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:36:07.231093 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d85713-a785-43d2-8bf1-ac18f65b0531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:07.231522 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d85713-a785-43d2-8bf1-ac18f65b0531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:07.231999 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d85713-a785-43d2-8bf1-ac18f65b0531 admin admin] PATH_INFO: `/v3/users/3252192f00384eceb044566e60b3533b` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:07.233285 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d85713-a785-43d2-8bf1-ac18f65b0531 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3252192f00384eceb044566e60b3533b)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:36:07.235978 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d85713-a785-43d2-8bf1-ac18f65b0531 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:36:07.321467 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a6d85713-a785-43d2-8bf1-ac18f65b0531 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3252192f00384eceb044566e60b3533b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:36:07.324307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a6d85713-a785-43d2-8bf1-ac18f65b0531 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3252192f00384eceb044566e60b3533b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:36:07.381553 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a6d85713-a785-43d2-8bf1-ac18f65b0531 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3252192f00384eceb044566e60b3533b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:36:07.411709 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a6d85713-a785-43d2-8bf1-ac18f65b0531 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3252192f00384eceb044566e60b3533b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:36:07.421960 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-a6d85713-a785-43d2-8bf1-ac18f65b0531 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3252192f00384eceb044566e60b3533b'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:36:07.429935 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1397/2793] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:36:07 2022] DELETE /identity/v3/users/3252192f00384eceb044566e60b3533b => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:36:07.908119 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9505561e-2c64-42a8-836a-bc0f594638c8 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:36:07.910309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-9505561e-2c64-42a8-836a-bc0f594638c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:36:07.923986 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9505561e-2c64-42a8-836a-bc0f594638c8 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:36:08.004679 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf98831-c63d-4f6d-b610-19c5cd2b5a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:36:08.006102 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf98831-c63d-4f6d-b610-19c5cd2b5a2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:08.007208 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf98831-c63d-4f6d-b610-19c5cd2b5a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:08.007724 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf98831-c63d-4f6d-b610-19c5cd2b5a2e admin admin] PATH_INFO: `/v3/projects/9b8bce2486aa400c9b279b9343758dc9` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:08.009812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf98831-c63d-4f6d-b610-19c5cd2b5a2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b8bce2486aa400c9b279b9343758dc9)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:36:08.013778 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf98831-c63d-4f6d-b610-19c5cd2b5a2e admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:36:08.033599 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe2c4f-1885-4cdb-afc9-7b4768d80fb1 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:08.034230 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe2c4f-1885-4cdb-afc9-7b4768d80fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:08.034678 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe2c4f-1885-4cdb-afc9-7b4768d80fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:08.071096 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-81fe2c4f-1885-4cdb-afc9-7b4768d80fb1 None None] MFA Rules not processed for user `f15ce0909d8848f48c53e4e3868e9cef`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:36:08.072911 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-81fe2c4f-1885-4cdb-afc9-7b4768d80fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:36:08.080230 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-0bf98831-c63d-4f6d-b610-19c5cd2b5a2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b8bce2486aa400c9b279b9343758dc9', 'request_id': 'req-0bf98831-c63d-4f6d-b610-19c5cd2b5a2e'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:36:08.087198 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-0bf98831-c63d-4f6d-b610-19c5cd2b5a2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b8bce2486aa400c9b279b9343758dc9', 'request_id': 'req-0bf98831-c63d-4f6d-b610-19c5cd2b5a2e'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:36:08.101593 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-0bf98831-c63d-4f6d-b610-19c5cd2b5a2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b8bce2486aa400c9b279b9343758dc9', 'request_id': 'req-0bf98831-c63d-4f6d-b610-19c5cd2b5a2e'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:36:08.116487 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-0bf98831-c63d-4f6d-b610-19c5cd2b5a2e admin admin] The token cache is being invalidate because project 9b8bce2486aa400c9b279b9343758dc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:36:08.116817 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-0bf98831-c63d-4f6d-b610-19c5cd2b5a2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:36:08.117449 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-0bf98831-c63d-4f6d-b610-19c5cd2b5a2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:36:08.119025 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1397/2794] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:36:07 2022] DELETE /identity/v3/projects/9b8bce2486aa400c9b279b9343758dc9 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:36:08.124572 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e87db-072f-4cd8-ba9b-ad24c7645275 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:08.124867 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e87db-072f-4cd8-ba9b-ad24c7645275 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:08.125145 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e87db-072f-4cd8-ba9b-ad24c7645275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:08.141165 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:36:08.147515 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-b62e87db-072f-4cd8-ba9b-ad24c7645275 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:36:08.150096 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-b62e87db-072f-4cd8-ba9b-ad24c7645275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:36:08.162166 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1398/2795] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:36:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:36:08.178468 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe2c4f-1885-4cdb-afc9-7b4768d80fb1 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:36:08.179873 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-81fe2c4f-1885-4cdb-afc9-7b4768d80fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:36:08.204340 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:36:08.227086 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1398/2796] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:36:08.234919 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516971a-98c1-4bbc-85a7-545fef81a23f None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:08.235207 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516971a-98c1-4bbc-85a7-545fef81a23f None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:08.235463 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516971a-98c1-4bbc-85a7-545fef81a23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:08.257055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-0516971a-98c1-4bbc-85a7-545fef81a23f None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:36:08.258262 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-0516971a-98c1-4bbc-85a7-545fef81a23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:36:08.267787 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1399/2797] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:36:08.271730 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0516971a-98c1-4bbc-85a7-545fef81a23f None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:36:08.281391 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0516971a-98c1-4bbc-85a7-545fef81a23f None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:36:08.286035 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462b870-e5e6-4d64-b2d4-a4687bb48a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:36:08.286638 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462b870-e5e6-4d64-b2d4-a4687bb48a8b admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:08.286910 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462b870-e5e6-4d64-b2d4-a4687bb48a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:08.287173 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462b870-e5e6-4d64-b2d4-a4687bb48a8b admin admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:08.287809 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b462b870-e5e6-4d64-b2d4-a4687bb48a8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:36:08.288132 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b462b870-e5e6-4d64-b2d4-a4687bb48a8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:36:08.289732 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b462b870-e5e6-4d64-b2d4-a4687bb48a8b admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:36:08.301210 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe2c4f-1885-4cdb-afc9-7b4768d80fb1 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:36:08.302271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1400/2798] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:36:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:36:08.306778 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462b870-e5e6-4d64-b2d4-a4687bb48a8b admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:36:08.307413 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6f615-8b1d-4199-a098-b51a8d4c33b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:36:08.308016 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6f615-8b1d-4199-a098-b51a8d4c33b8 service nova] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:08.308361 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6f615-8b1d-4199-a098-b51a8d4c33b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:08.308731 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6f615-8b1d-4199-a098-b51a8d4c33b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:08.311161 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-a4c6f615-8b1d-4199-a098-b51a8d4c33b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:36:08.322406 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462b870-e5e6-4d64-b2d4-a4687bb48a8b admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:36:08.327694 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f99da-1741-4a6b-87d8-4b8eb66380e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:36:08.328463 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f99da-1741-4a6b-87d8-4b8eb66380e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:08.328757 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f99da-1741-4a6b-87d8-4b8eb66380e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:08.329002 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f99da-1741-4a6b-87d8-4b8eb66380e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:08.329799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141f99da-1741-4a6b-87d8-4b8eb66380e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:36:08.331837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141f99da-1741-4a6b-87d8-4b8eb66380e7 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:36:08.367089 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1401/2799] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:36:08 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:36:08.371762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f99da-1741-4a6b-87d8-4b8eb66380e7 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:36:08.390286 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c6f615-8b1d-4199-a098-b51a8d4c33b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:36:08.392055 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c6f615-8b1d-4199-a098-b51a8d4c33b8 service nova] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:36:08.401341 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1399/2800] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:36:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:36:08.441990 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f99da-1741-4a6b-87d8-4b8eb66380e7 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:36:08.446729 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8316f-baea-4e20-87e5-8bb9f9af455d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:36:08.447896 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8316f-baea-4e20-87e5-8bb9f9af455d admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:08.448150 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8316f-baea-4e20-87e5-8bb9f9af455d admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:08.448425 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8316f-baea-4e20-87e5-8bb9f9af455d admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:08.449060 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e8316f-baea-4e20-87e5-8bb9f9af455d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:36:08.451238 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e8316f-baea-4e20-87e5-8bb9f9af455d admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:36:08.477955 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1402/2801] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:36:08 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:36:08.482236 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6f615-8b1d-4199-a098-b51a8d4c33b8 service nova] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:36:08.510798 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6f615-8b1d-4199-a098-b51a8d4c33b8 service nova] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:36:08.515568 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a99dc20-a9f1-40a5-be01-bf4715a2b649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:36:08.516297 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99dc20-a9f1-40a5-be01-bf4715a2b649 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:08.516544 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99dc20-a9f1-40a5-be01-bf4715a2b649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:08.516816 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99dc20-a9f1-40a5-be01-bf4715a2b649 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:08.517456 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a99dc20-a9f1-40a5-be01-bf4715a2b649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:36:08.519103 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a99dc20-a9f1-40a5-be01-bf4715a2b649 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:36:08.531583 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1400/2802] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:36:08.535503 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8316f-baea-4e20-87e5-8bb9f9af455d admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:36:08.548360 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8316f-baea-4e20-87e5-8bb9f9af455d admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:36:08.554308 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2919d-d3ac-4e49-b175-935e6e5b99fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:36:08.555031 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2919d-d3ac-4e49-b175-935e6e5b99fb admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:08.555316 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2919d-d3ac-4e49-b175-935e6e5b99fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:08.555682 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2919d-d3ac-4e49-b175-935e6e5b99fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:08.556790 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2919d-d3ac-4e49-b175-935e6e5b99fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:36:08.558592 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2919d-d3ac-4e49-b175-935e6e5b99fb admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:36:08.570705 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1403/2803] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:36:08.575187 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a99dc20-a9f1-40a5-be01-bf4715a2b649 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:36:08.586418 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a99dc20-a9f1-40a5-be01-bf4715a2b649 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:36:08.593766 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f253ad-7942-483d-88de-9bcad86058db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:36:08.594438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f253ad-7942-483d-88de-9bcad86058db admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:08.594714 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f253ad-7942-483d-88de-9bcad86058db admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:08.594993 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f253ad-7942-483d-88de-9bcad86058db admin admin] PATH_INFO: `/v3/projects/e749692b9683425791f92baca207aa04/users/2e1a8096652a43049a0512c7922c1cc1/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:08.614809 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f253ad-7942-483d-88de-9bcad86058db admin admin] RBAC: Authorizing `identity:create_grant(project_id=e749692b9683425791f92baca207aa04, user_id=2e1a8096652a43049a0512c7922c1cc1, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:36:08.617523 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f253ad-7942-483d-88de-9bcad86058db admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:36:08.637704 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1401/2804] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:36:08 2022] PUT /identity/v3/projects/e749692b9683425791f92baca207aa04/users/2e1a8096652a43049a0512c7922c1cc1/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:36:08.642737 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096f028-27f4-4d2a-a559-c9fedf9fdad3 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:08.643090 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096f028-27f4-4d2a-a559-c9fedf9fdad3 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:08.643427 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096f028-27f4-4d2a-a559-c9fedf9fdad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:08.664415 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-e096f028-27f4-4d2a-a559-c9fedf9fdad3 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:36:08.666047 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-e096f028-27f4-4d2a-a559-c9fedf9fdad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:36:08.711657 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:36:08.737022 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1404/2805] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:36:08.747828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f253ad-7942-483d-88de-9bcad86058db admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:36:08.750364 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-48f253ad-7942-483d-88de-9bcad86058db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:36:08.852236 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f253ad-7942-483d-88de-9bcad86058db admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:36:08.856153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2744e7-f4b0-4a60-a5db-fed90046f43e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:36:08.856890 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2744e7-f4b0-4a60-a5db-fed90046f43e service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:08.857563 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2744e7-f4b0-4a60-a5db-fed90046f43e service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:08.857563 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2744e7-f4b0-4a60-a5db-fed90046f43e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:08.868936 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2744e7-f4b0-4a60-a5db-fed90046f43e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:36:08.870526 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2744e7-f4b0-4a60-a5db-fed90046f43e service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:36:08.875130 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:36:08.883546 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1402/2806] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:36:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:36:09.630284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc9c62-d222-4c02-bd36-e12ac884f7a2 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:09.631613 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc9c62-d222-4c02-bd36-e12ac884f7a2 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:09.632002 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc9c62-d222-4c02-bd36-e12ac884f7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:09.666455 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-43fc9c62-d222-4c02-bd36-e12ac884f7a2 None None] MFA Rules not processed for user `2e1a8096652a43049a0512c7922c1cc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:36:09.668356 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-43fc9c62-d222-4c02-bd36-e12ac884f7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:36:09.779112 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1405/2807] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:36:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:36:09.790915 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2744e7-f4b0-4a60-a5db-fed90046f43e service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:36:09.793743 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-9e2744e7-f4b0-4a60-a5db-fed90046f43e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:36:09.853681 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44d664-b451-41cd-9265-bf7b17807f71 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:09.854014 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44d664-b451-41cd-9265-bf7b17807f71 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:09.854294 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44d664-b451-41cd-9265-bf7b17807f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:09.874172 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-0e44d664-b451-41cd-9265-bf7b17807f71 None None] MFA Rules not processed for user `6a7ac4048d0646f78c0d813fa0112a83`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:36:09.875424 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-0e44d664-b451-41cd-9265-bf7b17807f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:36:09.877809 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2744e7-f4b0-4a60-a5db-fed90046f43e service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:36:09.881638 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596af53-6e2d-4bb6-a157-c6e6c4d27716 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2db57fe3915446acb57b4f84ca353379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:36:09.882316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596af53-6e2d-4bb6-a157-c6e6c4d27716 service cinder] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:09.882658 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596af53-6e2d-4bb6-a157-c6e6c4d27716 service cinder] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:09.883026 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596af53-6e2d-4bb6-a157-c6e6c4d27716 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:09.892481 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596af53-6e2d-4bb6-a157-c6e6c4d27716 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:36:09.894011 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596af53-6e2d-4bb6-a157-c6e6c4d27716 service cinder] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:36:09.905875 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1403/2808] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:36:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:36:09.940192 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1406/2809] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:36:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 04:36:11.332892 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799c916-a678-4eeb-868e-11348a65405f None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:11.333435 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799c916-a678-4eeb-868e-11348a65405f None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:11.333435 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799c916-a678-4eeb-868e-11348a65405f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:11.353357 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-2799c916-a678-4eeb-868e-11348a65405f None None] MFA Rules not processed for user `2e1a8096652a43049a0512c7922c1cc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:36:11.354568 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-2799c916-a678-4eeb-868e-11348a65405f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:36:11.361634 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1404/2810] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:36:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:36:11.650103 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44d664-b451-41cd-9265-bf7b17807f71 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:36:11.660835 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44d664-b451-41cd-9265-bf7b17807f71 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:36:11.665168 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd37965-f7ab-4766-ba3c-15f3330d27b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:36:11.665885 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd37965-f7ab-4766-ba3c-15f3330d27b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:11.666753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd37965-f7ab-4766-ba3c-15f3330d27b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:11.667040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd37965-f7ab-4766-ba3c-15f3330d27b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:11.669159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-9cd37965-f7ab-4766-ba3c-15f3330d27b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:36:11.681274 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd37965-f7ab-4766-ba3c-15f3330d27b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:36:11.683603 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd37965-f7ab-4766-ba3c-15f3330d27b5 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:36:11.694491 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1407/2811] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:36:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:36:12.529228 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799c916-a678-4eeb-868e-11348a65405f None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:36:12.530624 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-2799c916-a678-4eeb-868e-11348a65405f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:36:12.650416 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799c916-a678-4eeb-868e-11348a65405f None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:36:12.656121 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c9c42-474e-435d-b0ef-7e86e4f37c81 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd503fe3cf6d428e962c14ab864ee608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:36:12.656796 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c9c42-474e-435d-b0ef-7e86e4f37c81 service placement] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:12.657281 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c9c42-474e-435d-b0ef-7e86e4f37c81 service placement] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:12.657760 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c9c42-474e-435d-b0ef-7e86e4f37c81 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:12.660836 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-db3c9c42-474e-435d-b0ef-7e86e4f37c81 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:36:12.674620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3c9c42-474e-435d-b0ef-7e86e4f37c81 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:36:12.676278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3c9c42-474e-435d-b0ef-7e86e4f37c81 service placement] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:36:12.687816 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1405/2812] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:36:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:36:21.689563 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=70 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:36:22.682457 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=71 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:36:25.696107 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c7e3d-b6ed-409c-a535-111e0b74dcc5 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:25.696661 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c7e3d-b6ed-409c-a535-111e0b74dcc5 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:25.696837 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c7e3d-b6ed-409c-a535-111e0b74dcc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:25.716122 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-661c7e3d-b6ed-409c-a535-111e0b74dcc5 None None] MFA Rules not processed for user `2e1a8096652a43049a0512c7922c1cc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:36:25.717636 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-661c7e3d-b6ed-409c-a535-111e0b74dcc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:36:25.725000 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1408/2813] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:36:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:36:35.711069 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=3 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:36:39.785285 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c9c42-474e-435d-b0ef-7e86e4f37c81 service placement] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:36:39.786915 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-db3c9c42-474e-435d-b0ef-7e86e4f37c81 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:36:39.799953 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c9c42-474e-435d-b0ef-7e86e4f37c81 service placement] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:36:39.803569 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ff208-cfa8-430e-81f6-459a6638dfcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:36:39.804228 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ff208-cfa8-430e-81f6-459a6638dfcc service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:39.804506 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ff208-cfa8-430e-81f6-459a6638dfcc service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:39.804785 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ff208-cfa8-430e-81f6-459a6638dfcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:39.880193 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ff208-cfa8-430e-81f6-459a6638dfcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:36:39.882186 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ff208-cfa8-430e-81f6-459a6638dfcc service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:36:39.895963 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1406/2814] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:36:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:36:45.006867 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24cde62-a011-48e4-92f3-5bd2f357c983 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:45.007359 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24cde62-a011-48e4-92f3-5bd2f357c983 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:45.007471 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24cde62-a011-48e4-92f3-5bd2f357c983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:45.038523 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-f24cde62-a011-48e4-92f3-5bd2f357c983 None None] MFA Rules not processed for user `d3538271ebfd41aabe9f3da85feb7bca`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:36:45.039615 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-f24cde62-a011-48e4-92f3-5bd2f357c983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:36:45.105559 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1409/2815] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Aug 23 04:36:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:36:45.153035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ff208-cfa8-430e-81f6-459a6638dfcc service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:36:45.154637 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-6c0ff208-cfa8-430e-81f6-459a6638dfcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:36:45.235479 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ff208-cfa8-430e-81f6-459a6638dfcc service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:36:45.241593 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2994ec4-642a-422d-8538-b9ce826b439d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f1418fd211a448da1a50175dce13b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:36:45.242204 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2994ec4-642a-422d-8538-b9ce826b439d service swift] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:45.242481 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2994ec4-642a-422d-8538-b9ce826b439d service swift] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:45.242762 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2994ec4-642a-422d-8538-b9ce826b439d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:45.251425 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2994ec4-642a-422d-8538-b9ce826b439d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:36:45.253037 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2994ec4-642a-422d-8538-b9ce826b439d service swift] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:36:45.261547 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1407/2816] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Aug 23 04:36:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 04:36:55.097700 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=29 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:36:55.257919 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=62 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:36:59.857806 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510741e3-b448-41c2-8ee4-56122ca33057 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:59.858338 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510741e3-b448-41c2-8ee4-56122ca33057 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:59.858338 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510741e3-b448-41c2-8ee4-56122ca33057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:59.879880 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-510741e3-b448-41c2-8ee4-56122ca33057 None None] MFA Rules not processed for user `d3538271ebfd41aabe9f3da85feb7bca`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:36:59.881097 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-510741e3-b448-41c2-8ee4-56122ca33057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:36:59.889498 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1410/2817] 10.222.0.18 () {58 vars in 967 bytes} [Tue Aug 23 04:36:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:36:59.905374 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2994ec4-642a-422d-8538-b9ce826b439d service swift] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:36:59.919607 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2994ec4-642a-422d-8538-b9ce826b439d service swift] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:36:59.926108 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f915e-cb7d-4fd3-af0b-d60c2cb2c530 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f1418fd211a448da1a50175dce13b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:36:59.926702 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f915e-cb7d-4fd3-af0b-d60c2cb2c530 service swift] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:36:59.926975 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f915e-cb7d-4fd3-af0b-d60c2cb2c530 service swift] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:36:59.927242 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f915e-cb7d-4fd3-af0b-d60c2cb2c530 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:36:59.935346 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f915e-cb7d-4fd3-af0b-d60c2cb2c530 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:36:59.936947 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f915e-cb7d-4fd3-af0b-d60c2cb2c530 service swift] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:36:59.945015 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1408/2818] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Aug 23 04:36:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 04:37:01.923829 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510741e3-b448-41c2-8ee4-56122ca33057 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:37:01.936440 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510741e3-b448-41c2-8ee4-56122ca33057 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:37:01.941175 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78cf50-4a57-4e08-a121-f2cbb8feae33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:37:01.941812 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78cf50-4a57-4e08-a121-f2cbb8feae33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:37:01.942090 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78cf50-4a57-4e08-a121-f2cbb8feae33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:37:01.942363 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78cf50-4a57-4e08-a121-f2cbb8feae33 admin admin] PATH_INFO: `/v3/users/2e1a8096652a43049a0512c7922c1cc1` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:37:01.943597 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b78cf50-4a57-4e08-a121-f2cbb8feae33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e1a8096652a43049a0512c7922c1cc1)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:37:01.946220 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b78cf50-4a57-4e08-a121-f2cbb8feae33 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:37:01.996065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-5b78cf50-4a57-4e08-a121-f2cbb8feae33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e1a8096652a43049a0512c7922c1cc1'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:37:02.005224 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-5b78cf50-4a57-4e08-a121-f2cbb8feae33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e1a8096652a43049a0512c7922c1cc1'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:37:02.019311 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-5b78cf50-4a57-4e08-a121-f2cbb8feae33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e1a8096652a43049a0512c7922c1cc1'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:37:02.020236 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-5b78cf50-4a57-4e08-a121-f2cbb8feae33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e1a8096652a43049a0512c7922c1cc1'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:37:02.030529 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-5b78cf50-4a57-4e08-a121-f2cbb8feae33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e1a8096652a43049a0512c7922c1cc1'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:37:02.037314 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1411/2819] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:37:01 2022] DELETE /identity/v3/users/2e1a8096652a43049a0512c7922c1cc1 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:37:02.216750 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f915e-cb7d-4fd3-af0b-d60c2cb2c530 service swift] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:37:02.229292 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f915e-cb7d-4fd3-af0b-d60c2cb2c530 service swift] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:37:02.295503 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce109cc-f890-44c6-a247-745aec8f6177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:37:02.296168 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce109cc-f890-44c6-a247-745aec8f6177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:37:02.296416 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce109cc-f890-44c6-a247-745aec8f6177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:37:02.296686 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce109cc-f890-44c6-a247-745aec8f6177 admin admin] PATH_INFO: `/v3/projects/e749692b9683425791f92baca207aa04` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:37:02.297797 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce109cc-f890-44c6-a247-745aec8f6177 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e749692b9683425791f92baca207aa04)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:37:02.299687 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce109cc-f890-44c6-a247-745aec8f6177 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:37:02.351960 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-3ce109cc-f890-44c6-a247-745aec8f6177 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e749692b9683425791f92baca207aa04', 'request_id': 'req-3ce109cc-f890-44c6-a247-745aec8f6177'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:37:02.358456 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-3ce109cc-f890-44c6-a247-745aec8f6177 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e749692b9683425791f92baca207aa04', 'request_id': 'req-3ce109cc-f890-44c6-a247-745aec8f6177'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:37:02.368500 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-3ce109cc-f890-44c6-a247-745aec8f6177 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e749692b9683425791f92baca207aa04', 'request_id': 'req-3ce109cc-f890-44c6-a247-745aec8f6177'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:37:02.382031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-3ce109cc-f890-44c6-a247-745aec8f6177 admin admin] The token cache is being invalidate because project e749692b9683425791f92baca207aa04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:37:02.382320 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-3ce109cc-f890-44c6-a247-745aec8f6177 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:37:02.382952 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-3ce109cc-f890-44c6-a247-745aec8f6177 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:37:02.384692 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1409/2820] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:37:02 2022] DELETE /identity/v3/projects/e749692b9683425791f92baca207aa04 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:37:02.390650 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebedb4b-7c91-4aec-947c-b434a292adbc None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:37:02.390974 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebedb4b-7c91-4aec-947c-b434a292adbc None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:37:02.391342 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebedb4b-7c91-4aec-947c-b434a292adbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:37:02.419771 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-2ebedb4b-7c91-4aec-947c-b434a292adbc None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:37:02.421049 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-2ebedb4b-7c91-4aec-947c-b434a292adbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:37:02.494593 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1412/2821] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:37:02.502538 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8acdd-160d-4ee2-b8df-8e2499158199 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:37:02.502825 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8acdd-160d-4ee2-b8df-8e2499158199 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:37:02.503099 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8acdd-160d-4ee2-b8df-8e2499158199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:37:02.524065 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-05d8acdd-160d-4ee2-b8df-8e2499158199 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:37:02.525292 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-05d8acdd-160d-4ee2-b8df-8e2499158199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:37:02.533619 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1410/2822] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:37:02.537959 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebedb4b-7c91-4aec-947c-b434a292adbc None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:37:02.548370 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebedb4b-7c91-4aec-947c-b434a292adbc None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:37:02.553483 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3c2b3-d036-4777-b01b-57c4840524c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:37:02.554084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3c2b3-d036-4777-b01b-57c4840524c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:37:02.554364 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3c2b3-d036-4777-b01b-57c4840524c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:37:02.554768 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3c2b3-d036-4777-b01b-57c4840524c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:37:02.555420 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3c2b3-d036-4777-b01b-57c4840524c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:37:02.555723 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3c2b3-d036-4777-b01b-57c4840524c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:37:02.557382 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3c2b3-d036-4777-b01b-57c4840524c9 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:37:02.569168 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1413/2823] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:37:02.576895 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d8acdd-160d-4ee2-b8df-8e2499158199 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:37:02.587764 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d8acdd-160d-4ee2-b8df-8e2499158199 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:37:02.593698 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faddf6b7-cbbe-4570-b71f-3994f41a23e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:37:02.594292 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faddf6b7-cbbe-4570-b71f-3994f41a23e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:37:02.594568 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faddf6b7-cbbe-4570-b71f-3994f41a23e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:37:02.594837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faddf6b7-cbbe-4570-b71f-3994f41a23e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:37:02.595455 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faddf6b7-cbbe-4570-b71f-3994f41a23e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:37:02.597396 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faddf6b7-cbbe-4570-b71f-3994f41a23e3 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:37:02.628125 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1411/2824] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:37:02 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:37:02.634001 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3c2b3-d036-4777-b01b-57c4840524c9 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:37:02.644159 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3c2b3-d036-4777-b01b-57c4840524c9 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:37:02.696234 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:37:02.707328 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273f38b-8db6-471d-8466-e09e5f212aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:37:02.708026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273f38b-8db6-471d-8466-e09e5f212aad admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:37:02.708332 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273f38b-8db6-471d-8466-e09e5f212aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:37:02.708600 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273f38b-8db6-471d-8466-e09e5f212aad admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:37:02.709239 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b273f38b-8db6-471d-8466-e09e5f212aad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:37:02.712294 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b273f38b-8db6-471d-8466-e09e5f212aad admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:37:02.737532 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1414/2825] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:37:02 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:37:02.743229 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faddf6b7-cbbe-4570-b71f-3994f41a23e3 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:37:02.754098 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faddf6b7-cbbe-4570-b71f-3994f41a23e3 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:37:02.759823 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fcb0f1-7cb9-4c94-a363-31c96cf3dd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:37:02.760459 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcb0f1-7cb9-4c94-a363-31c96cf3dd0f admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:37:02.760735 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcb0f1-7cb9-4c94-a363-31c96cf3dd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:37:02.760997 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcb0f1-7cb9-4c94-a363-31c96cf3dd0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:37:02.761633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fcb0f1-7cb9-4c94-a363-31c96cf3dd0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:37:02.763212 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fcb0f1-7cb9-4c94-a363-31c96cf3dd0f admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:37:02.773636 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1412/2826] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:37:02.779286 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273f38b-8db6-471d-8466-e09e5f212aad admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:37:02.790271 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273f38b-8db6-471d-8466-e09e5f212aad admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:37:02.795573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9056b2-82e8-408f-9c64-f1448edb2d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:37:02.796197 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9056b2-82e8-408f-9c64-f1448edb2d3b admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:37:02.796468 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9056b2-82e8-408f-9c64-f1448edb2d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:37:02.796735 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9056b2-82e8-408f-9c64-f1448edb2d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:37:02.797378 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9056b2-82e8-408f-9c64-f1448edb2d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:37:02.798963 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9056b2-82e8-408f-9c64-f1448edb2d3b admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:37:02.810049 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1415/2827] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:37:02.815224 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fcb0f1-7cb9-4c94-a363-31c96cf3dd0f admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:37:02.825605 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fcb0f1-7cb9-4c94-a363-31c96cf3dd0f admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:37:02.830472 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f746154b-c3de-48cd-b062-732855c12e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:37:02.831125 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746154b-c3de-48cd-b062-732855c12e37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:37:02.831383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746154b-c3de-48cd-b062-732855c12e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:37:02.831649 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746154b-c3de-48cd-b062-732855c12e37 admin admin] PATH_INFO: `/v3/projects/0111954f3e6e430387495701fa502227/users/7c1f40a97d2e42668e18961b8fd8a065/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:37:02.846270 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f746154b-c3de-48cd-b062-732855c12e37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0111954f3e6e430387495701fa502227, user_id=7c1f40a97d2e42668e18961b8fd8a065, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:37:02.848940 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f746154b-c3de-48cd-b062-732855c12e37 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:37:02.865991 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1413/2828] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:37:02 2022] PUT /identity/v3/projects/0111954f3e6e430387495701fa502227/users/7c1f40a97d2e42668e18961b8fd8a065/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:37:02.873099 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c86d72-b58a-436f-98fb-3df3fbda55e4 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:37:02.873463 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c86d72-b58a-436f-98fb-3df3fbda55e4 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:37:02.873795 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c86d72-b58a-436f-98fb-3df3fbda55e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:37:02.896771 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-b5c86d72-b58a-436f-98fb-3df3fbda55e4 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:37:02.897963 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-b5c86d72-b58a-436f-98fb-3df3fbda55e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:37:02.967954 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1416/2829] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:37:02.978151 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f746154b-c3de-48cd-b062-732855c12e37 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:37:02.980087 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-f746154b-c3de-48cd-b062-732855c12e37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:37:02.995345 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:37:03.046046 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f746154b-c3de-48cd-b062-732855c12e37 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:37:03.052069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e555005-0bc3-49d9-bc9e-648aa21ee831 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:37:03.052914 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e555005-0bc3-49d9-bc9e-648aa21ee831 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:37:03.053269 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e555005-0bc3-49d9-bc9e-648aa21ee831 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:37:03.053633 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e555005-0bc3-49d9-bc9e-648aa21ee831 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:37:03.064803 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e555005-0bc3-49d9-bc9e-648aa21ee831 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:37:03.066272 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e555005-0bc3-49d9-bc9e-648aa21ee831 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:37:03.076422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1414/2830] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:37:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:37:07.286853 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65328c7a-8d1d-438e-bf68-0ac27cf7d0e7 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:37:07.287752 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65328c7a-8d1d-438e-bf68-0ac27cf7d0e7 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:37:07.288065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65328c7a-8d1d-438e-bf68-0ac27cf7d0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:37:07.318729 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-65328c7a-8d1d-438e-bf68-0ac27cf7d0e7 None None] MFA Rules not processed for user `7c1f40a97d2e42668e18961b8fd8a065`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:37:07.320424 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-65328c7a-8d1d-438e-bf68-0ac27cf7d0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:37:07.386893 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1417/2831] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:37:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:37:07.391531 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e555005-0bc3-49d9-bc9e-648aa21ee831 service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:37:07.402319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e555005-0bc3-49d9-bc9e-648aa21ee831 service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:37:07.407443 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82074957-aff0-4311-8124-9e6b20984899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:37:07.408260 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82074957-aff0-4311-8124-9e6b20984899 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:37:07.408542 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82074957-aff0-4311-8124-9e6b20984899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:37:07.408813 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82074957-aff0-4311-8124-9e6b20984899 admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:37:07.409470 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82074957-aff0-4311-8124-9e6b20984899 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:37:07.411397 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82074957-aff0-4311-8124-9e6b20984899 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:37:07.443781 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1415/2832] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:37:07 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:37:07.448523 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65328c7a-8d1d-438e-bf68-0ac27cf7d0e7 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:37:07.460140 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65328c7a-8d1d-438e-bf68-0ac27cf7d0e7 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:37:07.523329 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19eec5-2023-405c-b9b6-43914af17820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:37:07.524084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19eec5-2023-405c-b9b6-43914af17820 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:37:07.524367 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19eec5-2023-405c-b9b6-43914af17820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:37:07.524641 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19eec5-2023-405c-b9b6-43914af17820 admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:37:07.525273 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad19eec5-2023-405c-b9b6-43914af17820 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:37:07.527219 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad19eec5-2023-405c-b9b6-43914af17820 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:37:07.536279 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:37:07.550527 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1418/2833] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:37:07 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:37:07.555032 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82074957-aff0-4311-8124-9e6b20984899 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:37:07.565706 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82074957-aff0-4311-8124-9e6b20984899 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:37:07.571540 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8905b-779e-4266-8bd7-b2f107f7937f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:37:07.572112 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8905b-779e-4266-8bd7-b2f107f7937f admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:37:07.572391 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8905b-779e-4266-8bd7-b2f107f7937f admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:37:07.572658 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8905b-779e-4266-8bd7-b2f107f7937f admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:37:07.573305 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8905b-779e-4266-8bd7-b2f107f7937f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:37:07.574911 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8905b-779e-4266-8bd7-b2f107f7937f admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:37:07.586305 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1416/2834] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:37:07.590997 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19eec5-2023-405c-b9b6-43914af17820 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:37:07.605280 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19eec5-2023-405c-b9b6-43914af17820 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:37:07.612121 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd270d-4cf7-465f-9ffc-27dba3e3c5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:37:07.612789 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd270d-4cf7-465f-9ffc-27dba3e3c5cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:37:07.613100 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd270d-4cf7-465f-9ffc-27dba3e3c5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:37:07.613525 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd270d-4cf7-465f-9ffc-27dba3e3c5cd admin admin] PATH_INFO: `/v3/projects/ff7ea9dab7cf446cbd299a4ac26b4668/users/59352c86952b4790b94c8da0ad73f405/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:37:07.631386 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd270d-4cf7-465f-9ffc-27dba3e3c5cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff7ea9dab7cf446cbd299a4ac26b4668, user_id=59352c86952b4790b94c8da0ad73f405, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:37:07.634124 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd270d-4cf7-465f-9ffc-27dba3e3c5cd admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:37:07.649545 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1419/2835] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:37:07 2022] PUT /identity/v3/projects/ff7ea9dab7cf446cbd299a4ac26b4668/users/59352c86952b4790b94c8da0ad73f405/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:37:07.653686 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8905b-779e-4266-8bd7-b2f107f7937f admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:37:07.664008 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8905b-779e-4266-8bd7-b2f107f7937f admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:37:07.723536 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3f47a-c47b-488b-9442-338699664ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:37:07.724167 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3f47a-c47b-488b-9442-338699664ef5 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:37:07.724441 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3f47a-c47b-488b-9442-338699664ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:37:07.724707 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3f47a-c47b-488b-9442-338699664ef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:37:07.725356 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b3f47a-c47b-488b-9442-338699664ef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:37:07.727428 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b3f47a-c47b-488b-9442-338699664ef5 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:37:07.737596 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1417/2836] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:37:07.742094 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd270d-4cf7-465f-9ffc-27dba3e3c5cd admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:37:07.753288 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd270d-4cf7-465f-9ffc-27dba3e3c5cd admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:37:07.758557 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6b2c7-96b5-4031-83ab-be82f15fe5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:37:07.759222 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6b2c7-96b5-4031-83ab-be82f15fe5a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:37:07.759498 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6b2c7-96b5-4031-83ab-be82f15fe5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:37:07.759766 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6b2c7-96b5-4031-83ab-be82f15fe5a7 admin admin] PATH_INFO: `/v3/projects/ff7ea9dab7cf446cbd299a4ac26b4668/users/59352c86952b4790b94c8da0ad73f405/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:37:07.761779 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6b2c7-96b5-4031-83ab-be82f15fe5a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff7ea9dab7cf446cbd299a4ac26b4668, user_id=59352c86952b4790b94c8da0ad73f405, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:37:07.764534 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6b2c7-96b5-4031-83ab-be82f15fe5a7 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:37:07.775351 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1420/2837] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:37:07 2022] PUT /identity/v3/projects/ff7ea9dab7cf446cbd299a4ac26b4668/users/59352c86952b4790b94c8da0ad73f405/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:37:12.181810 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb32bbb-f4a3-4559-90fc-ead4bf1c8684 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:37:12.182419 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb32bbb-f4a3-4559-90fc-ead4bf1c8684 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:37:12.182645 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb32bbb-f4a3-4559-90fc-ead4bf1c8684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:37:12.213326 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-adb32bbb-f4a3-4559-90fc-ead4bf1c8684 None None] MFA Rules not processed for user `59352c86952b4790b94c8da0ad73f405`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:37:12.214559 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-adb32bbb-f4a3-4559-90fc-ead4bf1c8684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:37:12.287663 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1418/2838] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:37:12 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:37:12.300081 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6b2c7-96b5-4031-83ab-be82f15fe5a7 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:37:12.301921 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-aea6b2c7-96b5-4031-83ab-be82f15fe5a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:37:12.363589 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6b2c7-96b5-4031-83ab-be82f15fe5a7 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:37:12.369760 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23787761-58a9-45e9-9066-ce5b7639640d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f1bf93cff6c46e0801bb827203cb5ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:37:12.370393 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23787761-58a9-45e9-9066-ce5b7639640d service glance] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:37:12.371579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23787761-58a9-45e9-9066-ce5b7639640d service glance] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:37:12.371853 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23787761-58a9-45e9-9066-ce5b7639640d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:37:12.441051 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23787761-58a9-45e9-9066-ce5b7639640d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:37:12.442816 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23787761-58a9-45e9-9066-ce5b7639640d service glance] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:37:12.453173 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1421/2839] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:37:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:37:12.582853 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed785a-c7ca-4840-aedf-b088674bf14b None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:37:12.583268 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed785a-c7ca-4840-aedf-b088674bf14b None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:37:12.583766 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed785a-c7ca-4840-aedf-b088674bf14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:37:12.592612 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:37:12.605489 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-dbed785a-c7ca-4840-aedf-b088674bf14b None None] MFA Rules not processed for user `d3538271ebfd41aabe9f3da85feb7bca`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:37:12.606728 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-dbed785a-c7ca-4840-aedf-b088674bf14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:37:12.665235 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1419/2840] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Aug 23 04:37:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:37:12.678005 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23787761-58a9-45e9-9066-ce5b7639640d service glance] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:37:12.684604 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-23787761-58a9-45e9-9066-ce5b7639640d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:37:12.764956 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23787761-58a9-45e9-9066-ce5b7639640d service glance] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:37:12.769932 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861dd54-86bc-44ea-89ab-0db1a163236e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f1418fd211a448da1a50175dce13b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:37:12.770575 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861dd54-86bc-44ea-89ab-0db1a163236e service swift] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:37:12.770854 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861dd54-86bc-44ea-89ab-0db1a163236e service swift] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:37:12.771095 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861dd54-86bc-44ea-89ab-0db1a163236e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:37:12.780555 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861dd54-86bc-44ea-89ab-0db1a163236e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:37:12.782155 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861dd54-86bc-44ea-89ab-0db1a163236e service swift] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:37:12.785770 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:37:12.791698 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1422/2841] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Aug 23 04:37:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 04:37:21.136641 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159a43e-bf78-4d7f-8c6d-050d91c2f30d None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:37:21.137217 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159a43e-bf78-4d7f-8c6d-050d91c2f30d None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:37:21.137217 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159a43e-bf78-4d7f-8c6d-050d91c2f30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:37:21.156837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-8159a43e-bf78-4d7f-8c6d-050d91c2f30d None None] MFA Rules not processed for user `7c1f40a97d2e42668e18961b8fd8a065`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:37:21.158029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-8159a43e-bf78-4d7f-8c6d-050d91c2f30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:37:21.165560 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1420/2842] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:37:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:37:22.797760 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:37:23.015631 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef88edc-a32f-4147-88e0-e4a543188b22 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:37:23.015982 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef88edc-a32f-4147-88e0-e4a543188b22 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:37:23.016177 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef88edc-a32f-4147-88e0-e4a543188b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:37:23.035238 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-0ef88edc-a32f-4147-88e0-e4a543188b22 None None] MFA Rules not processed for user `d3538271ebfd41aabe9f3da85feb7bca`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:37:23.036469 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-0ef88edc-a32f-4147-88e0-e4a543188b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:37:23.043886 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1423/2843] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Aug 23 04:37:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:37:23.058180 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159a43e-bf78-4d7f-8c6d-050d91c2f30d None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:37:23.072378 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159a43e-bf78-4d7f-8c6d-050d91c2f30d None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:37:23.077268 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0163bd2-1b85-4f64-add0-eece43646721 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f1418fd211a448da1a50175dce13b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:37:23.077884 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0163bd2-1b85-4f64-add0-eece43646721 service swift] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:37:23.078162 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0163bd2-1b85-4f64-add0-eece43646721 service swift] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:37:23.078433 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0163bd2-1b85-4f64-add0-eece43646721 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:37:23.087007 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0163bd2-1b85-4f64-add0-eece43646721 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:37:23.088661 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0163bd2-1b85-4f64-add0-eece43646721 service swift] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:37:23.097288 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1421/2844] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Aug 23 04:37:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 04:37:25.203553 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef88edc-a32f-4147-88e0-e4a543188b22 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:37:25.264284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef88edc-a32f-4147-88e0-e4a543188b22 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:37:25.268525 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac277e-c268-4421-82c1-cf75b32644cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:37:25.269683 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac277e-c268-4421-82c1-cf75b32644cf service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:37:25.270217 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac277e-c268-4421-82c1-cf75b32644cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:37:25.270657 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac277e-c268-4421-82c1-cf75b32644cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:37:25.273169 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-16ac277e-c268-4421-82c1-cf75b32644cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:37:25.350153 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac277e-c268-4421-82c1-cf75b32644cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:37:25.351884 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac277e-c268-4421-82c1-cf75b32644cf service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:37:25.363160 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1424/2845] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:37:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:37:28.932551 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0163bd2-1b85-4f64-add0-eece43646721 service swift] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:37:28.941957 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0163bd2-1b85-4f64-add0-eece43646721 service swift] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:37:28.945573 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8bc04b-ae8a-4ca2-a04c-d4c238269bd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:37:28.946234 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bc04b-ae8a-4ca2-a04c-d4c238269bd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:37:28.946488 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bc04b-ae8a-4ca2-a04c-d4c238269bd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:37:28.946752 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bc04b-ae8a-4ca2-a04c-d4c238269bd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:37:28.948527 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-ed8bc04b-ae8a-4ca2-a04c-d4c238269bd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:37:28.963278 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8bc04b-ae8a-4ca2-a04c-d4c238269bd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:37:28.964768 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8bc04b-ae8a-4ca2-a04c-d4c238269bd0 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:37:28.978294 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1422/2846] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:37:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:37:35.358337 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=57 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:37:38.972377 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=38 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:37:47.917976 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac277e-c268-4421-82c1-cf75b32644cf service neutron] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:37:47.926942 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac277e-c268-4421-82c1-cf75b32644cf service neutron] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:37:47.931592 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08519101-c3b7-4466-8f79-4b6061455241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:37:47.932228 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08519101-c3b7-4466-8f79-4b6061455241 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:37:47.932501 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08519101-c3b7-4466-8f79-4b6061455241 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:37:47.932766 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08519101-c3b7-4466-8f79-4b6061455241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:37:47.942104 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08519101-c3b7-4466-8f79-4b6061455241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:37:47.943540 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08519101-c3b7-4466-8f79-4b6061455241 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:37:47.952926 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1425/2847] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:37:47 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:37:49.635853 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8bc04b-ae8a-4ca2-a04c-d4c238269bd0 service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:37:49.639061 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-ed8bc04b-ae8a-4ca2-a04c-d4c238269bd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:37:49.652990 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8bc04b-ae8a-4ca2-a04c-d4c238269bd0 service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:37:49.657708 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397650a7-034a-4051-8f01-de9645826796 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:37:49.658383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397650a7-034a-4051-8f01-de9645826796 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:37:49.658663 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397650a7-034a-4051-8f01-de9645826796 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:37:49.659695 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397650a7-034a-4051-8f01-de9645826796 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:37:49.661587 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-397650a7-034a-4051-8f01-de9645826796 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:37:49.673323 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397650a7-034a-4051-8f01-de9645826796 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:37:49.674757 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397650a7-034a-4051-8f01-de9645826796 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:37:49.684912 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1423/2848] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:37:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:37:57.949137 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=6 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:37:59.681405 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=6 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:38:52.608970 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb49b6-e16d-4214-a450-398d6029dbc9 None None] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:38:52.609593 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb49b6-e16d-4214-a450-398d6029dbc9 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:38:52.609593 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb49b6-e16d-4214-a450-398d6029dbc9 None None] PATH_INFO: `/` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:38:52.611026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1426/2849] 10.222.0.18 () {54 vars in 921 bytes} [Tue Aug 23 04:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 04:38:52.616849 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2048708-bd93-4ea1-8e5c-a74de26aecac None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:38:52.617375 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2048708-bd93-4ea1-8e5c-a74de26aecac None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:38:52.617375 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2048708-bd93-4ea1-8e5c-a74de26aecac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:38:52.660741 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-c2048708-bd93-4ea1-8e5c-a74de26aecac None None] MFA Rules not processed for user `fdca9c48e5d740f6bda8c48072b54c0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:38:52.661937 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-c2048708-bd93-4ea1-8e5c-a74de26aecac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:38:52.668817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1424/2850] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Aug 23 04:38:52 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:38:52.681040 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb49b6-e16d-4214-a450-398d6029dbc9 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:38:52.692829 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb49b6-e16d-4214-a450-398d6029dbc9 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:38:52.696862 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609cf0b-8c5c-4be3-a75d-7e886c66c9df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:38:52.697464 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609cf0b-8c5c-4be3-a75d-7e886c66c9df service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:38:52.697769 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609cf0b-8c5c-4be3-a75d-7e886c66c9df service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:38:52.698039 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609cf0b-8c5c-4be3-a75d-7e886c66c9df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:38:52.705493 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609cf0b-8c5c-4be3-a75d-7e886c66c9df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:38:52.706961 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609cf0b-8c5c-4be3-a75d-7e886c66c9df service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:38:52.715445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1427/2851] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:38:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:38:52.803754 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2a931-b8dd-4a34-90f9-8aced94f838e None None] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:38:52.804075 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2a931-b8dd-4a34-90f9-8aced94f838e None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:38:52.804359 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2a931-b8dd-4a34-90f9-8aced94f838e None None] PATH_INFO: `/` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:38:52.805726 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1425/2852] 10.222.0.18 () {54 vars in 921 bytes} [Tue Aug 23 04:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 04:38:52.812969 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b10348-179d-4021-90c1-3d43a07fd7a1 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:38:52.813231 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b10348-179d-4021-90c1-3d43a07fd7a1 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:38:52.813506 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b10348-179d-4021-90c1-3d43a07fd7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:38:52.851853 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-93b10348-179d-4021-90c1-3d43a07fd7a1 None None] MFA Rules not processed for user `cd7b11f479a54adc8d9b8641859b07a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:38:52.853021 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-93b10348-179d-4021-90c1-3d43a07fd7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:38:52.863267 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1428/2853] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Aug 23 04:38:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:38:52.875383 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2a931-b8dd-4a34-90f9-8aced94f838e None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:38:52.887609 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2a931-b8dd-4a34-90f9-8aced94f838e None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:38:52.891283 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c534273-be23-4d23-88fc-4df64e0a78d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:38:52.891857 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c534273-be23-4d23-88fc-4df64e0a78d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:38:52.892166 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c534273-be23-4d23-88fc-4df64e0a78d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:38:52.892429 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c534273-be23-4d23-88fc-4df64e0a78d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:38:52.903773 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c534273-be23-4d23-88fc-4df64e0a78d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:38:52.905241 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c534273-be23-4d23-88fc-4df64e0a78d3 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:38:52.916657 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1426/2854] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:38:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:38:52.990610 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b10348-179d-4021-90c1-3d43a07fd7a1 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:38:52.992141 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-93b10348-179d-4021-90c1-3d43a07fd7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:38:53.076108 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b10348-179d-4021-90c1-3d43a07fd7a1 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:38:53.082080 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da46717c-0dce-4af7-b49e-5d74a2a3a267 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd503fe3cf6d428e962c14ab864ee608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:38:53.082693 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da46717c-0dce-4af7-b49e-5d74a2a3a267 service placement] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:38:53.082966 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da46717c-0dce-4af7-b49e-5d74a2a3a267 service placement] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:38:53.083238 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da46717c-0dce-4af7-b49e-5d74a2a3a267 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:38:53.085155 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-da46717c-0dce-4af7-b49e-5d74a2a3a267 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:38:53.096947 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da46717c-0dce-4af7-b49e-5d74a2a3a267 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:38:53.098534 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da46717c-0dce-4af7-b49e-5d74a2a3a267 service placement] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:38:53.109195 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1429/2855] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:38:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:38:53.791416 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c534273-be23-4d23-88fc-4df64e0a78d3 service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:38:53.794043 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-5c534273-be23-4d23-88fc-4df64e0a78d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:38:53.818381 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c534273-be23-4d23-88fc-4df64e0a78d3 service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:38:53.828078 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d02f781-83e4-4d77-8866-7d41c466c316 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:38:53.829103 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02f781-83e4-4d77-8866-7d41c466c316 service nova] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:38:53.830073 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02f781-83e4-4d77-8866-7d41c466c316 service nova] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:38:53.830652 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02f781-83e4-4d77-8866-7d41c466c316 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:38:53.833614 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-9d02f781-83e4-4d77-8866-7d41c466c316 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:38:53.850880 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d02f781-83e4-4d77-8866-7d41c466c316 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:38:53.853035 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d02f781-83e4-4d77-8866-7d41c466c316 service nova] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:38:53.868146 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1427/2856] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:38:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:38:54.630251 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da46717c-0dce-4af7-b49e-5d74a2a3a267 service placement] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:38:54.648807 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da46717c-0dce-4af7-b49e-5d74a2a3a267 service placement] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:38:54.665791 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52371c03-93fd-48e3-8e6a-6e7abe3ec009 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:38:54.666945 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52371c03-93fd-48e3-8e6a-6e7abe3ec009 service nova] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:38:54.667445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52371c03-93fd-48e3-8e6a-6e7abe3ec009 service nova] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:38:54.667968 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52371c03-93fd-48e3-8e6a-6e7abe3ec009 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:38:54.672536 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-52371c03-93fd-48e3-8e6a-6e7abe3ec009 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:38:54.701101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52371c03-93fd-48e3-8e6a-6e7abe3ec009 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:38:54.702946 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52371c03-93fd-48e3-8e6a-6e7abe3ec009 service nova] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:38:54.718130 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1430/2857] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:38:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:38:55.345594 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d02f781-83e4-4d77-8866-7d41c466c316 service nova] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:38:55.347005 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-9d02f781-83e4-4d77-8866-7d41c466c316 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:38:55.363529 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d02f781-83e4-4d77-8866-7d41c466c316 service nova] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:38:55.369664 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e3644-7ce3-443e-9196-cd67548e0ebb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:38:55.371231 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e3644-7ce3-443e-9196-cd67548e0ebb service nova] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:38:55.371513 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e3644-7ce3-443e-9196-cd67548e0ebb service nova] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:38:55.371789 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e3644-7ce3-443e-9196-cd67548e0ebb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:38:55.373597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-058e3644-7ce3-443e-9196-cd67548e0ebb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:38:55.387557 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e3644-7ce3-443e-9196-cd67548e0ebb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:38:55.388997 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e3644-7ce3-443e-9196-cd67548e0ebb service nova] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:38:55.401244 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1428/2858] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:38:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:39:01.821016 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0001450-8d65-4c5f-910e-80be778db0a6 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:39:01.821563 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0001450-8d65-4c5f-910e-80be778db0a6 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:39:01.821563 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0001450-8d65-4c5f-910e-80be778db0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:39:01.875208 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-c0001450-8d65-4c5f-910e-80be778db0a6 None None] MFA Rules not processed for user `d3538271ebfd41aabe9f3da85feb7bca`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:39:01.876429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-c0001450-8d65-4c5f-910e-80be778db0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:39:01.894691 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1431/2859] 10.222.0.18 () {58 vars in 967 bytes} [Tue Aug 23 04:39:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:39:01.936575 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e3644-7ce3-443e-9196-cd67548e0ebb service nova] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:39:01.958242 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e3644-7ce3-443e-9196-cd67548e0ebb service nova] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:39:01.963546 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786bba83-b3ee-407e-baa2-d062c6850bc7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f1418fd211a448da1a50175dce13b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:39:01.964207 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bba83-b3ee-407e-baa2-d062c6850bc7 service swift] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:39:01.964482 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bba83-b3ee-407e-baa2-d062c6850bc7 service swift] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:39:01.964756 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bba83-b3ee-407e-baa2-d062c6850bc7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:39:01.972974 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786bba83-b3ee-407e-baa2-d062c6850bc7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:39:01.974469 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786bba83-b3ee-407e-baa2-d062c6850bc7 service swift] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:39:01.982901 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1429/2860] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Aug 23 04:39:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 04:39:05.469200 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0001450-8d65-4c5f-910e-80be778db0a6 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:39:05.471579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-c0001450-8d65-4c5f-910e-80be778db0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:39:05.488149 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0001450-8d65-4c5f-910e-80be778db0a6 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:39:05.494084 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb68b0d-50d2-46e2-8c01-f3d0043ff195 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd503fe3cf6d428e962c14ab864ee608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:39:05.494865 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb68b0d-50d2-46e2-8c01-f3d0043ff195 service placement] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:39:05.495235 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb68b0d-50d2-46e2-8c01-f3d0043ff195 service placement] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:39:05.495573 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb68b0d-50d2-46e2-8c01-f3d0043ff195 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:39:05.497636 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-bbb68b0d-50d2-46e2-8c01-f3d0043ff195 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:39:05.511733 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb68b0d-50d2-46e2-8c01-f3d0043ff195 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:39:05.513488 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb68b0d-50d2-46e2-8c01-f3d0043ff195 service placement] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:39:05.527272 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1432/2861] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:39:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:39:08.770921 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786bba83-b3ee-407e-baa2-d062c6850bc7 service swift] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:39:08.832964 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786bba83-b3ee-407e-baa2-d062c6850bc7 service swift] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:39:08.837560 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c397f-63ef-4786-9441-c9628f3e8550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:39:08.838349 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c397f-63ef-4786-9441-c9628f3e8550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:39:08.838632 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c397f-63ef-4786-9441-c9628f3e8550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:39:08.838909 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c397f-63ef-4786-9441-c9628f3e8550 admin admin] PATH_INFO: `/v3/users/7c1f40a97d2e42668e18961b8fd8a065` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:39:08.840031 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2c397f-63ef-4786-9441-c9628f3e8550 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c1f40a97d2e42668e18961b8fd8a065)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:39:08.842009 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2c397f-63ef-4786-9441-c9628f3e8550 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:39:08.902602 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-7c2c397f-63ef-4786-9441-c9628f3e8550 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c1f40a97d2e42668e18961b8fd8a065'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:39:08.903252 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-7c2c397f-63ef-4786-9441-c9628f3e8550 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c1f40a97d2e42668e18961b8fd8a065'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:39:08.912134 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-7c2c397f-63ef-4786-9441-c9628f3e8550 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c1f40a97d2e42668e18961b8fd8a065'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:39:08.924266 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-7c2c397f-63ef-4786-9441-c9628f3e8550 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c1f40a97d2e42668e18961b8fd8a065'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:39:08.931090 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-7c2c397f-63ef-4786-9441-c9628f3e8550 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c1f40a97d2e42668e18961b8fd8a065'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:39:08.932635 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:39:08.938135 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1430/2862] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:39:08 2022] DELETE /identity/v3/users/7c1f40a97d2e42668e18961b8fd8a065 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:39:09.118345 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb68b0d-50d2-46e2-8c01-f3d0043ff195 service placement] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:39:09.129801 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb68b0d-50d2-46e2-8c01-f3d0043ff195 service placement] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:39:09.179431 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=100 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:39:09.190579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67320288-200d-4404-a5cb-8b7add1c5d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:39:09.191217 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67320288-200d-4404-a5cb-8b7add1c5d5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:39:09.191500 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67320288-200d-4404-a5cb-8b7add1c5d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:39:09.191763 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67320288-200d-4404-a5cb-8b7add1c5d5f admin admin] PATH_INFO: `/v3/projects/0111954f3e6e430387495701fa502227` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:39:09.192922 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67320288-200d-4404-a5cb-8b7add1c5d5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0111954f3e6e430387495701fa502227)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:39:09.194821 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67320288-200d-4404-a5cb-8b7add1c5d5f admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:39:09.242909 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-67320288-200d-4404-a5cb-8b7add1c5d5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0111954f3e6e430387495701fa502227', 'request_id': 'req-67320288-200d-4404-a5cb-8b7add1c5d5f'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:39:09.249030 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-67320288-200d-4404-a5cb-8b7add1c5d5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0111954f3e6e430387495701fa502227', 'request_id': 'req-67320288-200d-4404-a5cb-8b7add1c5d5f'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:39:09.257362 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-67320288-200d-4404-a5cb-8b7add1c5d5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0111954f3e6e430387495701fa502227', 'request_id': 'req-67320288-200d-4404-a5cb-8b7add1c5d5f'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:39:09.267720 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-67320288-200d-4404-a5cb-8b7add1c5d5f admin admin] The token cache is being invalidate because project 0111954f3e6e430387495701fa502227 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:39:09.268037 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-67320288-200d-4404-a5cb-8b7add1c5d5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:39:09.268664 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-67320288-200d-4404-a5cb-8b7add1c5d5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:39:09.270223 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1433/2863] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:39:09 2022] DELETE /identity/v3/projects/0111954f3e6e430387495701fa502227 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:39:09.274582 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c397f-63ef-4786-9441-c9628f3e8550 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:39:09.276280 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-7c2c397f-63ef-4786-9441-c9628f3e8550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:39:09.290018 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c397f-63ef-4786-9441-c9628f3e8550 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:39:09.346601 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001ed38-24a4-4d1f-85c8-0d6bf5c9a494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:39:09.347252 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001ed38-24a4-4d1f-85c8-0d6bf5c9a494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:39:09.347510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001ed38-24a4-4d1f-85c8-0d6bf5c9a494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:39:09.347776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001ed38-24a4-4d1f-85c8-0d6bf5c9a494 admin admin] PATH_INFO: `/v3/users/59352c86952b4790b94c8da0ad73f405` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:39:09.348959 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001ed38-24a4-4d1f-85c8-0d6bf5c9a494 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59352c86952b4790b94c8da0ad73f405)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:39:09.351168 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001ed38-24a4-4d1f-85c8-0d6bf5c9a494 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:39:09.404061 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-4001ed38-24a4-4d1f-85c8-0d6bf5c9a494 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59352c86952b4790b94c8da0ad73f405'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:39:09.404790 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-4001ed38-24a4-4d1f-85c8-0d6bf5c9a494 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59352c86952b4790b94c8da0ad73f405'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:39:09.413408 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-4001ed38-24a4-4d1f-85c8-0d6bf5c9a494 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59352c86952b4790b94c8da0ad73f405'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:39:09.425559 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-4001ed38-24a4-4d1f-85c8-0d6bf5c9a494 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59352c86952b4790b94c8da0ad73f405'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:39:09.432829 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-4001ed38-24a4-4d1f-85c8-0d6bf5c9a494 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59352c86952b4790b94c8da0ad73f405'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:39:09.438875 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1431/2864] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:39:09 2022] DELETE /identity/v3/users/59352c86952b4790b94c8da0ad73f405 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:39:09.617945 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67320288-200d-4404-a5cb-8b7add1c5d5f admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:39:09.621515 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-67320288-200d-4404-a5cb-8b7add1c5d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:39:09.634385 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67320288-200d-4404-a5cb-8b7add1c5d5f admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:39:09.697403 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a27a9-8ea4-4871-bab3-6a44253ef9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:39:09.698340 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a27a9-8ea4-4871-bab3-6a44253ef9e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:39:09.698621 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a27a9-8ea4-4871-bab3-6a44253ef9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:39:09.698895 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a27a9-8ea4-4871-bab3-6a44253ef9e2 admin admin] PATH_INFO: `/v3/projects/ff7ea9dab7cf446cbd299a4ac26b4668` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:39:09.700141 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a27a9-8ea4-4871-bab3-6a44253ef9e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff7ea9dab7cf446cbd299a4ac26b4668)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:39:09.702141 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a27a9-8ea4-4871-bab3-6a44253ef9e2 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:39:09.762701 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-521a27a9-8ea4-4871-bab3-6a44253ef9e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff7ea9dab7cf446cbd299a4ac26b4668', 'request_id': 'req-521a27a9-8ea4-4871-bab3-6a44253ef9e2'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:39:09.770322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-521a27a9-8ea4-4871-bab3-6a44253ef9e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff7ea9dab7cf446cbd299a4ac26b4668', 'request_id': 'req-521a27a9-8ea4-4871-bab3-6a44253ef9e2'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:39:09.783789 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-521a27a9-8ea4-4871-bab3-6a44253ef9e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff7ea9dab7cf446cbd299a4ac26b4668', 'request_id': 'req-521a27a9-8ea4-4871-bab3-6a44253ef9e2'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:39:09.796768 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-521a27a9-8ea4-4871-bab3-6a44253ef9e2 admin admin] The token cache is being invalidate because project ff7ea9dab7cf446cbd299a4ac26b4668 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:39:09.796973 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-521a27a9-8ea4-4871-bab3-6a44253ef9e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:39:09.797625 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-521a27a9-8ea4-4871-bab3-6a44253ef9e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:39:09.800791 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1434/2865] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:39:09 2022] DELETE /identity/v3/projects/ff7ea9dab7cf446cbd299a4ac26b4668 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:39:19.435511 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=54,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:39:19.794934 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=79,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:40:28.004431 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001ed38-24a4-4d1f-85c8-0d6bf5c9a494 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:40:28.006050 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-4001ed38-24a4-4d1f-85c8-0d6bf5c9a494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:40:28.072320 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a27a9-8ea4-4871-bab3-6a44253ef9e2 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:40:28.074181 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-521a27a9-8ea4-4871-bab3-6a44253ef9e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:40:28.094582 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001ed38-24a4-4d1f-85c8-0d6bf5c9a494 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:40:28.143564 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a27a9-8ea4-4871-bab3-6a44253ef9e2 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:40:28.148454 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1685fd2-b89d-43d9-992c-a4a21e8f4311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:40:28.149127 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1685fd2-b89d-43d9-992c-a4a21e8f4311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:40:28.149639 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1685fd2-b89d-43d9-992c-a4a21e8f4311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:40:28.149784 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1685fd2-b89d-43d9-992c-a4a21e8f4311 admin admin] PATH_INFO: `/v3/users/6a7ac4048d0646f78c0d813fa0112a83` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:40:28.150791 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1685fd2-b89d-43d9-992c-a4a21e8f4311 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a7ac4048d0646f78c0d813fa0112a83)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:40:28.152777 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1685fd2-b89d-43d9-992c-a4a21e8f4311 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:40:28.158975 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd70740-3e99-4344-b0ab-442fe7aebeff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd503fe3cf6d428e962c14ab864ee608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:40:28.159597 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd70740-3e99-4344-b0ab-442fe7aebeff service placement] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:40:28.159877 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd70740-3e99-4344-b0ab-442fe7aebeff service placement] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:40:28.160668 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd70740-3e99-4344-b0ab-442fe7aebeff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:40:28.162764 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-2cd70740-3e99-4344-b0ab-442fe7aebeff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:40:28.174298 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd70740-3e99-4344-b0ab-442fe7aebeff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:40:28.176524 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd70740-3e99-4344-b0ab-442fe7aebeff service placement] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:40:28.188116 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1432/2866] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:40:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:40:28.203104 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d1685fd2-b89d-43d9-992c-a4a21e8f4311 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a7ac4048d0646f78c0d813fa0112a83'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:40:28.212105 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d1685fd2-b89d-43d9-992c-a4a21e8f4311 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a7ac4048d0646f78c0d813fa0112a83'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:40:28.225090 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d1685fd2-b89d-43d9-992c-a4a21e8f4311 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a7ac4048d0646f78c0d813fa0112a83'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:40:28.225780 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d1685fd2-b89d-43d9-992c-a4a21e8f4311 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a7ac4048d0646f78c0d813fa0112a83'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:40:28.232704 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-d1685fd2-b89d-43d9-992c-a4a21e8f4311 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a7ac4048d0646f78c0d813fa0112a83'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:40:28.238544 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1435/2867] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:40:28 2022] DELETE /identity/v3/users/6a7ac4048d0646f78c0d813fa0112a83 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:40:28.419363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd70740-3e99-4344-b0ab-442fe7aebeff service placement] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:40:28.432950 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd70740-3e99-4344-b0ab-442fe7aebeff service placement] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:40:28.495600 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca8405-be9f-4a55-8609-cca40aa51f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:40:28.496360 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca8405-be9f-4a55-8609-cca40aa51f45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:40:28.496651 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca8405-be9f-4a55-8609-cca40aa51f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:40:28.496916 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca8405-be9f-4a55-8609-cca40aa51f45 admin admin] PATH_INFO: `/v3/projects/0fa7aa0063804717a00a59ea3e188779` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:40:28.498075 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca8405-be9f-4a55-8609-cca40aa51f45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fa7aa0063804717a00a59ea3e188779)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:40:28.500029 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca8405-be9f-4a55-8609-cca40aa51f45 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:40:28.541498 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:40:28.552418 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-4bca8405-be9f-4a55-8609-cca40aa51f45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fa7aa0063804717a00a59ea3e188779', 'request_id': 'req-4bca8405-be9f-4a55-8609-cca40aa51f45'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:40:28.558731 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-4bca8405-be9f-4a55-8609-cca40aa51f45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fa7aa0063804717a00a59ea3e188779', 'request_id': 'req-4bca8405-be9f-4a55-8609-cca40aa51f45'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:40:28.566510 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-4bca8405-be9f-4a55-8609-cca40aa51f45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fa7aa0063804717a00a59ea3e188779', 'request_id': 'req-4bca8405-be9f-4a55-8609-cca40aa51f45'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:40:28.577225 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-4bca8405-be9f-4a55-8609-cca40aa51f45 admin admin] The token cache is being invalidate because project 0fa7aa0063804717a00a59ea3e188779 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:40:28.577363 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-4bca8405-be9f-4a55-8609-cca40aa51f45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:40:28.577986 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-4bca8405-be9f-4a55-8609-cca40aa51f45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:40:28.580418 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1433/2868] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:40:28 2022] DELETE /identity/v3/projects/0fa7aa0063804717a00a59ea3e188779 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:40:28.583933 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1685fd2-b89d-43d9-992c-a4a21e8f4311 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:40:28.585649 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-d1685fd2-b89d-43d9-992c-a4a21e8f4311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:40:28.600855 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1685fd2-b89d-43d9-992c-a4a21e8f4311 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:40:28.660537 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993655e0-ff06-4773-b64e-18219442f689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:40:28.661164 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993655e0-ff06-4773-b64e-18219442f689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:40:28.661438 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993655e0-ff06-4773-b64e-18219442f689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:40:28.661705 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993655e0-ff06-4773-b64e-18219442f689 admin admin] PATH_INFO: `/v3/users/f15ce0909d8848f48c53e4e3868e9cef` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:40:28.663013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993655e0-ff06-4773-b64e-18219442f689 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f15ce0909d8848f48c53e4e3868e9cef)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:40:28.664912 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993655e0-ff06-4773-b64e-18219442f689 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:40:28.686445 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:40:28.721553 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-993655e0-ff06-4773-b64e-18219442f689 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f15ce0909d8848f48c53e4e3868e9cef'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:40:28.730727 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-993655e0-ff06-4773-b64e-18219442f689 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f15ce0909d8848f48c53e4e3868e9cef'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:40:28.743642 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-993655e0-ff06-4773-b64e-18219442f689 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f15ce0909d8848f48c53e4e3868e9cef'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:40:28.744328 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-993655e0-ff06-4773-b64e-18219442f689 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f15ce0909d8848f48c53e4e3868e9cef'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:40:28.751429 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-993655e0-ff06-4773-b64e-18219442f689 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f15ce0909d8848f48c53e4e3868e9cef'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:40:28.757481 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1436/2869] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:40:28 2022] DELETE /identity/v3/users/f15ce0909d8848f48c53e4e3868e9cef => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:40:28.938747 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca8405-be9f-4a55-8609-cca40aa51f45 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:40:28.940300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-4bca8405-be9f-4a55-8609-cca40aa51f45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:40:28.953277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca8405-be9f-4a55-8609-cca40aa51f45 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:40:29.013817 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7d48d-8a4c-4a79-b5eb-734f7b347b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:40:29.014536 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7d48d-8a4c-4a79-b5eb-734f7b347b93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:40:29.014725 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7d48d-8a4c-4a79-b5eb-734f7b347b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:40:29.014999 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7d48d-8a4c-4a79-b5eb-734f7b347b93 admin admin] PATH_INFO: `/v3/projects/e283a55c05814e32a0cbd231bfa2a4e0` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:40:29.016131 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf7d48d-8a4c-4a79-b5eb-734f7b347b93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e283a55c05814e32a0cbd231bfa2a4e0)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:40:29.018055 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf7d48d-8a4c-4a79-b5eb-734f7b347b93 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:40:29.065013 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-aaf7d48d-8a4c-4a79-b5eb-734f7b347b93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e283a55c05814e32a0cbd231bfa2a4e0', 'request_id': 'req-aaf7d48d-8a4c-4a79-b5eb-734f7b347b93'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:40:29.071316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-aaf7d48d-8a4c-4a79-b5eb-734f7b347b93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e283a55c05814e32a0cbd231bfa2a4e0', 'request_id': 'req-aaf7d48d-8a4c-4a79-b5eb-734f7b347b93'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:40:29.080698 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-aaf7d48d-8a4c-4a79-b5eb-734f7b347b93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e283a55c05814e32a0cbd231bfa2a4e0', 'request_id': 'req-aaf7d48d-8a4c-4a79-b5eb-734f7b347b93'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:40:29.095636 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-aaf7d48d-8a4c-4a79-b5eb-734f7b347b93 admin admin] The token cache is being invalidate because project e283a55c05814e32a0cbd231bfa2a4e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:40:29.095949 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-aaf7d48d-8a4c-4a79-b5eb-734f7b347b93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:40:29.096773 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-aaf7d48d-8a4c-4a79-b5eb-734f7b347b93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:40:29.101120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1434/2870] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:40:28 2022] DELETE /identity/v3/projects/e283a55c05814e32a0cbd231bfa2a4e0 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:40:29.108162 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080f7e5-c658-48dc-bf5e-e77788f165c3 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:40:29.108668 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080f7e5-c658-48dc-bf5e-e77788f165c3 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:40:29.108891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080f7e5-c658-48dc-bf5e-e77788f165c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:40:29.135492 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-f080f7e5-c658-48dc-bf5e-e77788f165c3 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:40:29.136841 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-f080f7e5-c658-48dc-bf5e-e77788f165c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:40:29.201395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1437/2871] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:40:29.209606 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab746ba-122f-40b0-8e94-5627e855d809 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:40:29.209886 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab746ba-122f-40b0-8e94-5627e855d809 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:40:29.210188 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab746ba-122f-40b0-8e94-5627e855d809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:40:29.232074 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-4ab746ba-122f-40b0-8e94-5627e855d809 None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:40:29.233274 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-4ab746ba-122f-40b0-8e94-5627e855d809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:40:29.243431 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1435/2872] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:40:29.248309 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080f7e5-c658-48dc-bf5e-e77788f165c3 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:40:29.258704 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080f7e5-c658-48dc-bf5e-e77788f165c3 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:40:29.264504 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2955a02-e539-4149-b4db-c236f3ca796a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:40:29.265110 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2955a02-e539-4149-b4db-c236f3ca796a admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:40:29.265387 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2955a02-e539-4149-b4db-c236f3ca796a admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:40:29.265658 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2955a02-e539-4149-b4db-c236f3ca796a admin admin] PATH_INFO: `/v3/domains` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:40:29.266311 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2955a02-e539-4149-b4db-c236f3ca796a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 04:40:29.266614 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2955a02-e539-4149-b4db-c236f3ca796a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:40:29.268264 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2955a02-e539-4149-b4db-c236f3ca796a admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:40:29.280268 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1438/2873] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Aug 23 04:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 04:40:29.284866 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab746ba-122f-40b0-8e94-5627e855d809 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:40:29.294307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab746ba-122f-40b0-8e94-5627e855d809 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:40:29.299370 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc9eb0-700b-4eb7-8d1f-bbc6ec35c865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:40:29.299961 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc9eb0-700b-4eb7-8d1f-bbc6ec35c865 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:40:29.300268 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc9eb0-700b-4eb7-8d1f-bbc6ec35c865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:40:29.300744 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc9eb0-700b-4eb7-8d1f-bbc6ec35c865 admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:40:29.303438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc9eb0-700b-4eb7-8d1f-bbc6ec35c865 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:40:29.303438 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc9eb0-700b-4eb7-8d1f-bbc6ec35c865 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:40:29.348562 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1436/2874] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:40:29 2022] POST /identity/v3/projects => generated 377 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 04:40:29.354287 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2955a02-e539-4149-b4db-c236f3ca796a admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:40:29.371039 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2955a02-e539-4149-b4db-c236f3ca796a admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:40:29.447232 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400d294-5345-463d-ba80-d0d5c4c21395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:40:29.447844 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400d294-5345-463d-ba80-d0d5c4c21395 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:40:29.448157 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400d294-5345-463d-ba80-d0d5c4c21395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:40:29.448430 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400d294-5345-463d-ba80-d0d5c4c21395 admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:40:29.449048 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400d294-5345-463d-ba80-d0d5c4c21395 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:40:29.450981 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400d294-5345-463d-ba80-d0d5c4c21395 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:40:29.474960 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1439/2875] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:40:29 2022] POST /identity/v3/users => generated 289 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:40:29.479125 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc9eb0-700b-4eb7-8d1f-bbc6ec35c865 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:40:29.489725 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc9eb0-700b-4eb7-8d1f-bbc6ec35c865 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:40:29.494715 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532909b-bdec-4e7d-8f15-717b3f637676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:40:29.495319 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532909b-bdec-4e7d-8f15-717b3f637676 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:40:29.495595 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532909b-bdec-4e7d-8f15-717b3f637676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:40:29.495863 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532909b-bdec-4e7d-8f15-717b3f637676 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:40:29.500932 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0532909b-bdec-4e7d-8f15-717b3f637676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:40:29.503153 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0532909b-bdec-4e7d-8f15-717b3f637676 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:40:29.514511 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1437/2876] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:40:29.520657 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400d294-5345-463d-ba80-d0d5c4c21395 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:40:29.539043 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400d294-5345-463d-ba80-d0d5c4c21395 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:40:29.545145 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9922a04-7aa6-4f39-abc2-e6aba5d41a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:40:29.545773 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9922a04-7aa6-4f39-abc2-e6aba5d41a49 admin admin] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:40:29.546055 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9922a04-7aa6-4f39-abc2-e6aba5d41a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:40:29.546306 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9922a04-7aa6-4f39-abc2-e6aba5d41a49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:40:29.546984 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9922a04-7aa6-4f39-abc2-e6aba5d41a49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:40:29.548635 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9922a04-7aa6-4f39-abc2-e6aba5d41a49 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:40:29.559616 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1440/2877] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 04:40:29.563940 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532909b-bdec-4e7d-8f15-717b3f637676 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:40:29.570669 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:40:29.574693 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532909b-bdec-4e7d-8f15-717b3f637676 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:40:29.581542 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcdb377-81f6-4fda-990c-eb7747e5034d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:40:29.582629 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcdb377-81f6-4fda-990c-eb7747e5034d admin admin] REQUEST_METHOD: `PUT` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:40:29.583154 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcdb377-81f6-4fda-990c-eb7747e5034d admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:40:29.583554 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcdb377-81f6-4fda-990c-eb7747e5034d admin admin] PATH_INFO: `/v3/projects/65e721c90c8d41088ec9f9d92dd5858d/users/041280e59ffa4e518a599442fe9ded41/roles/aa03504b4fb642d288d07f4335e291e1` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:40:29.600454 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcdb377-81f6-4fda-990c-eb7747e5034d admin admin] RBAC: Authorizing `identity:create_grant(project_id=65e721c90c8d41088ec9f9d92dd5858d, user_id=041280e59ffa4e518a599442fe9ded41, role_id=aa03504b4fb642d288d07f4335e291e1)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:40:29.603166 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcdb377-81f6-4fda-990c-eb7747e5034d admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:40:29.618829 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1438/2878] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:40:29 2022] PUT /identity/v3/projects/65e721c90c8d41088ec9f9d92dd5858d/users/041280e59ffa4e518a599442fe9ded41/roles/aa03504b4fb642d288d07f4335e291e1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:40:29.624158 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8b640-d735-491c-ab35-fe0cf5abc58e None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:40:29.624449 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8b640-d735-491c-ab35-fe0cf5abc58e None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:40:29.624728 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8b640-d735-491c-ab35-fe0cf5abc58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:40:29.633364 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:40:29.648727 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-abc8b640-d735-491c-ab35-fe0cf5abc58e None None] MFA Rules not processed for user `de65eb39f13f4a92a6a526fb5ab1f5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:40:29.651485 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-abc8b640-d735-491c-ab35-fe0cf5abc58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:40:29.715435 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1441/2879] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:40:29.725967 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcdb377-81f6-4fda-990c-eb7747e5034d admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:40:29.727774 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-ddcdb377-81f6-4fda-990c-eb7747e5034d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:40:29.789972 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcdb377-81f6-4fda-990c-eb7747e5034d admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:40:29.794300 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfc2a6-a2a0-4f95-b5b7-bb2da847f7d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:40:29.794910 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfc2a6-a2a0-4f95-b5b7-bb2da847f7d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:40:29.795186 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfc2a6-a2a0-4f95-b5b7-bb2da847f7d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:40:29.795459 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfc2a6-a2a0-4f95-b5b7-bb2da847f7d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:40:29.805190 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dfc2a6-a2a0-4f95-b5b7-bb2da847f7d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:40:29.806630 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dfc2a6-a2a0-4f95-b5b7-bb2da847f7d5 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:40:29.816454 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1439/2880] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:40:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:40:33.844322 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06801da-8d43-47b6-bf18-5ab0b2c4de12 None None] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:40:33.844892 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06801da-8d43-47b6-bf18-5ab0b2c4de12 None None] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:40:33.844892 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06801da-8d43-47b6-bf18-5ab0b2c4de12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:40:33.875094 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.auth.core [None req-d06801da-8d43-47b6-bf18-5ab0b2c4de12 None None] MFA Rules not processed for user `041280e59ffa4e518a599442fe9ded41`. Rule list: `[]` (Enabled: `True`). {{(pid=81934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:40:33.876428 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-d06801da-8d43-47b6-bf18-5ab0b2c4de12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:40:33.948057 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1442/2881] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:40:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:40:33.952032 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfc2a6-a2a0-4f95-b5b7-bb2da847f7d5 service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:40:33.963307 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfc2a6-a2a0-4f95-b5b7-bb2da847f7d5 service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:40:33.968808 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f1a1d7-7364-49c8-b201-f30962eb3518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:40:33.969430 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1a1d7-7364-49c8-b201-f30962eb3518 admin admin] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:40:33.969813 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1a1d7-7364-49c8-b201-f30962eb3518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:40:33.969972 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1a1d7-7364-49c8-b201-f30962eb3518 admin admin] PATH_INFO: `/v3/projects` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:40:33.970598 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f1a1d7-7364-49c8-b201-f30962eb3518 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:40:33.972560 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f1a1d7-7364-49c8-b201-f30962eb3518 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:40:34.003277 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1440/2882] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Aug 23 04:40:33 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:40:34.007735 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06801da-8d43-47b6-bf18-5ab0b2c4de12 None None] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:40:34.018350 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06801da-8d43-47b6-bf18-5ab0b2c4de12 None None] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:40:34.080245 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0399f53-e422-44ad-9c92-6796449eeff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:40:34.081083 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0399f53-e422-44ad-9c92-6796449eeff8 admin admin] REQUEST_METHOD: `POST` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:40:34.081461 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0399f53-e422-44ad-9c92-6796449eeff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:40:34.081856 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0399f53-e422-44ad-9c92-6796449eeff8 admin admin] PATH_INFO: `/v3/users` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:40:34.082532 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0399f53-e422-44ad-9c92-6796449eeff8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:40:34.084502 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0399f53-e422-44ad-9c92-6796449eeff8 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:40:34.107758 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1443/2883] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Aug 23 04:40:34 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 04:40:34.111897 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f1a1d7-7364-49c8-b201-f30962eb3518 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:40:34.122707 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f1a1d7-7364-49c8-b201-f30962eb3518 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:40:34.127464 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f4b4f8-504a-42a5-9e6f-66953899aae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:40:34.128095 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f4b4f8-504a-42a5-9e6f-66953899aae6 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:40:34.128373 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f4b4f8-504a-42a5-9e6f-66953899aae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:40:34.128683 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f4b4f8-504a-42a5-9e6f-66953899aae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:40:34.129421 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f4b4f8-504a-42a5-9e6f-66953899aae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:40:34.131022 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f4b4f8-504a-42a5-9e6f-66953899aae6 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:40:34.141786 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1441/2884] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:40:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:40:34.145865 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0399f53-e422-44ad-9c92-6796449eeff8 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:40:34.156310 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0399f53-e422-44ad-9c92-6796449eeff8 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:40:34.161287 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d0334-56f9-43d3-be34-39aae7455a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:40:34.161989 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d0334-56f9-43d3-be34-39aae7455a20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:40:34.162267 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d0334-56f9-43d3-be34-39aae7455a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:40:34.162540 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d0334-56f9-43d3-be34-39aae7455a20 admin admin] PATH_INFO: `/v3/projects/c229879e107d4348ac054e1a7b5e8e95/users/7ad61ef7d5454658a8d7d044c7441d56/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:40:34.178968 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2d0334-56f9-43d3-be34-39aae7455a20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c229879e107d4348ac054e1a7b5e8e95, user_id=7ad61ef7d5454658a8d7d044c7441d56, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:40:34.181948 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2d0334-56f9-43d3-be34-39aae7455a20 admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:40:34.193020 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:40:34.197891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1444/2885] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:40:34 2022] PUT /identity/v3/projects/c229879e107d4348ac054e1a7b5e8e95/users/7ad61ef7d5454658a8d7d044c7441d56/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:40:34.202133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f4b4f8-504a-42a5-9e6f-66953899aae6 admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:40:34.212496 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f4b4f8-504a-42a5-9e6f-66953899aae6 admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:40:34.273784 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5a437-5898-489d-aed9-b6204fdf9eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:40:34.274306 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5a437-5898-489d-aed9-b6204fdf9eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:40:34.274580 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5a437-5898-489d-aed9-b6204fdf9eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:40:34.274848 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5a437-5898-489d-aed9-b6204fdf9eb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:40:34.275493 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5a437-5898-489d-aed9-b6204fdf9eb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:40:34.277106 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5a437-5898-489d-aed9-b6204fdf9eb6 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:40:34.288333 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1442/2886] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Aug 23 04:40:34 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 04:40:34.292680 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d0334-56f9-43d3-be34-39aae7455a20 admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:40:34.304126 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d0334-56f9-43d3-be34-39aae7455a20 admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:40:34.309283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256bbfa-1fdd-4cbe-94d4-f7832f76db0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:40:34.309943 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256bbfa-1fdd-4cbe-94d4-f7832f76db0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:40:34.310212 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256bbfa-1fdd-4cbe-94d4-f7832f76db0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:40:34.310483 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256bbfa-1fdd-4cbe-94d4-f7832f76db0d admin admin] PATH_INFO: `/v3/projects/c229879e107d4348ac054e1a7b5e8e95/users/7ad61ef7d5454658a8d7d044c7441d56/roles/52b31061b72c45ccb76dd26bf31f8d3f` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:40:34.312570 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256bbfa-1fdd-4cbe-94d4-f7832f76db0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c229879e107d4348ac054e1a7b5e8e95, user_id=7ad61ef7d5454658a8d7d044c7441d56, role_id=52b31061b72c45ccb76dd26bf31f8d3f)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:40:34.315219 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256bbfa-1fdd-4cbe-94d4-f7832f76db0d admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:40:34.325839 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1445/2887] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Aug 23 04:40:34 2022] PUT /identity/v3/projects/c229879e107d4348ac054e1a7b5e8e95/users/7ad61ef7d5454658a8d7d044c7441d56/roles/52b31061b72c45ccb76dd26bf31f8d3f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 04:40:38.541118 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28ec79-bfee-47fc-86c1-9dd09e522dd2 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:40:38.541931 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28ec79-bfee-47fc-86c1-9dd09e522dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:40:38.542380 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28ec79-bfee-47fc-86c1-9dd09e522dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:40:38.586397 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-9c28ec79-bfee-47fc-86c1-9dd09e522dd2 None None] MFA Rules not processed for user `7ad61ef7d5454658a8d7d044c7441d56`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:40:38.587784 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-9c28ec79-bfee-47fc-86c1-9dd09e522dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:40:38.632514 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:40:38.676598 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1443/2888] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:40:38 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:40:38.695303 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256bbfa-1fdd-4cbe-94d4-f7832f76db0d admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:40:38.697638 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-8256bbfa-1fdd-4cbe-94d4-f7832f76db0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:40:38.787968 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256bbfa-1fdd-4cbe-94d4-f7832f76db0d admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:40:38.793714 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f50fcf-fa41-4c82-ba97-a269332aa7c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:40:38.794454 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f50fcf-fa41-4c82-ba97-a269332aa7c7 service nova] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:40:38.794799 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f50fcf-fa41-4c82-ba97-a269332aa7c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:40:38.795139 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f50fcf-fa41-4c82-ba97-a269332aa7c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:40:38.859482 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f50fcf-fa41-4c82-ba97-a269332aa7c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:40:38.864781 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f50fcf-fa41-4c82-ba97-a269332aa7c7 service nova] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:40:38.880169 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1446/2889] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:40:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:40:48.673107 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=13 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:40:48.874339 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=61,keystone:INSERT=1 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:40:54.585883 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720b2fc-db17-44a9-ae08-2a9784906b9f None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:40:54.586427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720b2fc-db17-44a9-ae08-2a9784906b9f None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:40:54.586427 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720b2fc-db17-44a9-ae08-2a9784906b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:40:54.611973 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-7720b2fc-db17-44a9-ae08-2a9784906b9f None None] MFA Rules not processed for user `041280e59ffa4e518a599442fe9ded41`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:40:54.613244 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-7720b2fc-db17-44a9-ae08-2a9784906b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:40:54.621026 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1444/2890] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:40:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:40:54.950102 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f50fcf-fa41-4c82-ba97-a269332aa7c7 service nova] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:40:55.014351 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f50fcf-fa41-4c82-ba97-a269332aa7c7 service nova] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:40:55.018230 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd354cf-e7e8-4953-8516-7cff8c4b8f1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:40:55.018832 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd354cf-e7e8-4953-8516-7cff8c4b8f1b service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:40:55.019103 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd354cf-e7e8-4953-8516-7cff8c4b8f1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:40:55.019365 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd354cf-e7e8-4953-8516-7cff8c4b8f1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:40:55.021249 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-4fd354cf-e7e8-4953-8516-7cff8c4b8f1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:40:55.032586 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd354cf-e7e8-4953-8516-7cff8c4b8f1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:40:55.034238 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd354cf-e7e8-4953-8516-7cff8c4b8f1b service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:40:55.043127 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1447/2891] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:40:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:41:04.606673 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=3 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:41:05.038911 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=26 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:41:11.219253 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720b2fc-db17-44a9-ae08-2a9784906b9f None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:41:11.220831 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-7720b2fc-db17-44a9-ae08-2a9784906b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:41:11.234696 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720b2fc-db17-44a9-ae08-2a9784906b9f None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:41:11.239211 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18f2a8-1536-4ca2-831e-f9ecce95b7bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:41:11.239811 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18f2a8-1536-4ca2-831e-f9ecce95b7bf service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:41:11.240120 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18f2a8-1536-4ca2-831e-f9ecce95b7bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:41:11.240386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18f2a8-1536-4ca2-831e-f9ecce95b7bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:41:11.254037 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a18f2a8-1536-4ca2-831e-f9ecce95b7bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:41:11.255855 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a18f2a8-1536-4ca2-831e-f9ecce95b7bf service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:41:11.269746 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1445/2892] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:41:11 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:41:21.264154 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=6 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:41:32.005434 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd354cf-e7e8-4953-8516-7cff8c4b8f1b service neutron] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:41:32.007621 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-4fd354cf-e7e8-4953-8516-7cff8c4b8f1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:41:32.093387 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd354cf-e7e8-4953-8516-7cff8c4b8f1b service neutron] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:41:32.099416 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e08448-b716-44fc-bf91-1905fd5c9725 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd503fe3cf6d428e962c14ab864ee608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:41:32.100077 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e08448-b716-44fc-bf91-1905fd5c9725 service placement] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:41:32.100359 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e08448-b716-44fc-bf91-1905fd5c9725 service placement] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:41:32.100628 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e08448-b716-44fc-bf91-1905fd5c9725 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:41:32.102660 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-42e08448-b716-44fc-bf91-1905fd5c9725 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:41:32.115580 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e08448-b716-44fc-bf91-1905fd5c9725 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:41:32.117200 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e08448-b716-44fc-bf91-1905fd5c9725 service placement] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:41:32.127785 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1448/2893] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:41:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:41:42.123256 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=32 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:43:08.686837 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18f2a8-1536-4ca2-831e-f9ecce95b7bf service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:43:08.696259 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18f2a8-1536-4ca2-831e-f9ecce95b7bf service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:43:08.700027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22494d-529d-4c2a-9360-e634cbcd875b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:43:08.700625 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22494d-529d-4c2a-9360-e634cbcd875b service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:43:08.700898 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22494d-529d-4c2a-9360-e634cbcd875b service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:43:08.701165 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22494d-529d-4c2a-9360-e634cbcd875b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:43:08.702998 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-ed22494d-529d-4c2a-9360-e634cbcd875b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:43:08.717800 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22494d-529d-4c2a-9360-e634cbcd875b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:43:08.719140 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22494d-529d-4c2a-9360-e634cbcd875b service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:43:08.732062 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1446/2894] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:43:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:43:09.922972 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e08448-b716-44fc-bf91-1905fd5c9725 service placement] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:43:09.933486 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e08448-b716-44fc-bf91-1905fd5c9725 service placement] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:43:09.938691 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c15ac69-b2fb-42b4-8cc5-2fbaef8c46d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:43:09.939300 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15ac69-b2fb-42b4-8cc5-2fbaef8c46d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:43:09.939579 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15ac69-b2fb-42b4-8cc5-2fbaef8c46d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:43:09.939849 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15ac69-b2fb-42b4-8cc5-2fbaef8c46d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:43:09.942277 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-0c15ac69-b2fb-42b4-8cc5-2fbaef8c46d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:43:09.957769 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c15ac69-b2fb-42b4-8cc5-2fbaef8c46d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:43:09.959233 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c15ac69-b2fb-42b4-8cc5-2fbaef8c46d3 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:43:09.973920 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1449/2895] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:43:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:43:12.265154 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc36f4c-5789-4980-9b26-16777a5ac670 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:43:12.265670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc36f4c-5789-4980-9b26-16777a5ac670 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:43:12.265670 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc36f4c-5789-4980-9b26-16777a5ac670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:43:12.288219 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-5cc36f4c-5789-4980-9b26-16777a5ac670 None None] MFA Rules not processed for user `041280e59ffa4e518a599442fe9ded41`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:43:12.289422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-5cc36f4c-5789-4980-9b26-16777a5ac670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:43:12.296776 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1447/2896] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:43:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:43:14.766311 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c15ac69-b2fb-42b4-8cc5-2fbaef8c46d3 service neutron] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:43:14.776100 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c15ac69-b2fb-42b4-8cc5-2fbaef8c46d3 service neutron] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:43:14.780125 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff21e2c-5038-4bf0-9fe7-0db4bb64cd3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:43:14.780752 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff21e2c-5038-4bf0-9fe7-0db4bb64cd3f service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:43:14.781029 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff21e2c-5038-4bf0-9fe7-0db4bb64cd3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:43:14.781298 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff21e2c-5038-4bf0-9fe7-0db4bb64cd3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:43:14.783161 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-0ff21e2c-5038-4bf0-9fe7-0db4bb64cd3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:43:14.796071 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff21e2c-5038-4bf0-9fe7-0db4bb64cd3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:43:14.797658 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff21e2c-5038-4bf0-9fe7-0db4bb64cd3f service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:43:14.806891 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1450/2897] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:43:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:43:22.283093 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=9 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:43:24.802822 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=12 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:45:06.449287 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533faa1a-8eda-4065-8d72-4208813e5249 None None] REQUEST_METHOD: `POST` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:45:06.449845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533faa1a-8eda-4065-8d72-4208813e5249 None None] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:45:06.449845 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533faa1a-8eda-4065-8d72-4208813e5249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:45:06.469655 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.auth.core [None req-533faa1a-8eda-4065-8d72-4208813e5249 None None] MFA Rules not processed for user `041280e59ffa4e518a599442fe9ded41`. Rule list: `[]` (Enabled: `True`). {{(pid=81933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 04:45:06.470873 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-533faa1a-8eda-4065-8d72-4208813e5249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:45:06.478568 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1448/2898] 10.222.0.18 () {58 vars in 958 bytes} [Tue Aug 23 04:45:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 04:45:06.824162 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff21e2c-5038-4bf0-9fe7-0db4bb64cd3f service neutron] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:45:06.833473 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff21e2c-5038-4bf0-9fe7-0db4bb64cd3f service neutron] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:45:06.838186 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1e10c-d8db-4304-89d3-ec3d94881076 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:45:06.838793 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1e10c-d8db-4304-89d3-ec3d94881076 service neutron] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:45:06.839056 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1e10c-d8db-4304-89d3-ec3d94881076 service neutron] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:45:06.839323 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1e10c-d8db-4304-89d3-ec3d94881076 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:45:06.841426 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-2fb1e10c-d8db-4304-89d3-ec3d94881076 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:45:06.852939 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb1e10c-d8db-4304-89d3-ec3d94881076 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:45:06.854395 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb1e10c-d8db-4304-89d3-ec3d94881076 service neutron] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:45:06.863626 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1451/2899] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:45:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:45:08.378347 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533faa1a-8eda-4065-8d72-4208813e5249 None None] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:45:08.380021 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-533faa1a-8eda-4065-8d72-4208813e5249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:45:08.396166 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533faa1a-8eda-4065-8d72-4208813e5249 None None] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:45:08.402030 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468943c9-40ca-4308-a143-07a120e1a520 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd503fe3cf6d428e962c14ab864ee608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:45:08.402735 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468943c9-40ca-4308-a143-07a120e1a520 service placement] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:45:08.403033 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468943c9-40ca-4308-a143-07a120e1a520 service placement] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:45:08.403281 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468943c9-40ca-4308-a143-07a120e1a520 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:45:08.405181 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-468943c9-40ca-4308-a143-07a120e1a520 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:45:08.416629 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468943c9-40ca-4308-a143-07a120e1a520 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:45:08.418187 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468943c9-40ca-4308-a143-07a120e1a520 service placement] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:45:08.429027 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1449/2900] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:45:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:45:13.787115 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1e10c-d8db-4304-89d3-ec3d94881076 service neutron] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:45:13.788795 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-2fb1e10c-d8db-4304-89d3-ec3d94881076 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:45:13.807251 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1e10c-d8db-4304-89d3-ec3d94881076 service neutron] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:45:13.812854 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc028e-a8bb-4515-aad6-aa7903a1567d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:45:13.813465 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc028e-a8bb-4515-aad6-aa7903a1567d service nova] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:45:13.813736 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc028e-a8bb-4515-aad6-aa7903a1567d service nova] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:45:13.814002 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc028e-a8bb-4515-aad6-aa7903a1567d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:45:13.815743 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-3bdc028e-a8bb-4515-aad6-aa7903a1567d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:45:13.828362 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc028e-a8bb-4515-aad6-aa7903a1567d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:45:13.829945 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc028e-a8bb-4515-aad6-aa7903a1567d service nova] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:45:13.840826 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1452/2901] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:45:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:45:18.424284 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=9 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:45:23.836053 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=12 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:45:24.447553 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468943c9-40ca-4308-a143-07a120e1a520 service placement] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:45:24.461985 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468943c9-40ca-4308-a143-07a120e1a520 service placement] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:45:24.470316 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be48d8a-18c9-4a37-b0ee-4c2ef8d9271b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:45:24.470963 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be48d8a-18c9-4a37-b0ee-4c2ef8d9271b service nova] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:45:24.471232 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be48d8a-18c9-4a37-b0ee-4c2ef8d9271b service nova] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:45:24.471518 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be48d8a-18c9-4a37-b0ee-4c2ef8d9271b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:45:24.473821 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-1be48d8a-18c9-4a37-b0ee-4c2ef8d9271b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:45:24.486993 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be48d8a-18c9-4a37-b0ee-4c2ef8d9271b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:45:24.488636 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be48d8a-18c9-4a37-b0ee-4c2ef8d9271b service nova] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:45:24.500187 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1450/2902] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:45:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:45:34.495376 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=6 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:46:36.032307 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc028e-a8bb-4515-aad6-aa7903a1567d service nova] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:46:36.110678 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc028e-a8bb-4515-aad6-aa7903a1567d service nova] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:46:36.117286 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b4148-4e4b-4652-9863-0767bd4aa642 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd503fe3cf6d428e962c14ab864ee608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:46:36.117903 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b4148-4e4b-4652-9863-0767bd4aa642 service placement] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:46:36.118210 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b4148-4e4b-4652-9863-0767bd4aa642 service placement] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:46:36.118482 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b4148-4e4b-4652-9863-0767bd4aa642 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:46:36.127751 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02b4148-4e4b-4652-9863-0767bd4aa642 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:46:36.129391 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02b4148-4e4b-4652-9863-0767bd4aa642 service placement] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:46:36.140026 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1453/2903] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:46:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:46:46.135548 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=27 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:46:58.537620 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be48d8a-18c9-4a37-b0ee-4c2ef8d9271b service nova] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:46:58.579730 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be48d8a-18c9-4a37-b0ee-4c2ef8d9271b service nova] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:46:58.583895 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f85c39-630d-44fd-b9d7-da176705ed75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:46:58.584531 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f85c39-630d-44fd-b9d7-da176705ed75 service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:46:58.584813 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f85c39-630d-44fd-b9d7-da176705ed75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:46:58.585079 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f85c39-630d-44fd-b9d7-da176705ed75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:46:58.593895 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f85c39-630d-44fd-b9d7-da176705ed75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:46:58.595321 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f85c39-630d-44fd-b9d7-da176705ed75 service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:46:58.604595 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1451/2904] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:46:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:47:01.855815 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b4148-4e4b-4652-9863-0767bd4aa642 service placement] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:47:01.889208 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b4148-4e4b-4652-9863-0767bd4aa642 service placement] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:47:01.894950 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b334bb0-b44d-4186-9914-94a2bc95c6e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7b11f479a54adc8d9b8641859b07a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:47:01.895571 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b334bb0-b44d-4186-9914-94a2bc95c6e5 service nova] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:47:01.895850 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b334bb0-b44d-4186-9914-94a2bc95c6e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:47:01.896158 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b334bb0-b44d-4186-9914-94a2bc95c6e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:47:01.897886 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-8b334bb0-b44d-4186-9914-94a2bc95c6e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:47:01.933247 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b334bb0-b44d-4186-9914-94a2bc95c6e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:47:01.934886 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b334bb0-b44d-4186-9914-94a2bc95c6e5 service nova] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:47:01.946440 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1454/2905] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:47:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 04:47:08.600461 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=18 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:47:11.941398 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=22 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:47:12.162638 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f85c39-630d-44fd-b9d7-da176705ed75 service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:47:12.172211 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f85c39-630d-44fd-b9d7-da176705ed75 service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:47:12.176574 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8270769-27fa-4519-bbc9-d7f662a26d4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdca9c48e5d740f6bda8c48072b54c0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:47:12.177191 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8270769-27fa-4519-bbc9-d7f662a26d4c service neutron] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:47:12.177477 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8270769-27fa-4519-bbc9-d7f662a26d4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:47:12.177751 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8270769-27fa-4519-bbc9-d7f662a26d4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:47:12.254327 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8270769-27fa-4519-bbc9-d7f662a26d4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:47:12.255808 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8270769-27fa-4519-bbc9-d7f662a26d4c service neutron] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:47:12.275281 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1452/2906] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:47:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:47:15.468540 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b334bb0-b44d-4186-9914-94a2bc95c6e5 service nova] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:47:15.488825 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b334bb0-b44d-4186-9914-94a2bc95c6e5 service nova] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:47:15.494192 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f90a80-4553-4d56-8c07-588390f7cba4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd503fe3cf6d428e962c14ab864ee608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:47:15.494838 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f90a80-4553-4d56-8c07-588390f7cba4 service placement] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:47:15.495283 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f90a80-4553-4d56-8c07-588390f7cba4 service placement] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:47:15.495762 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f90a80-4553-4d56-8c07-588390f7cba4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:47:15.497745 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-f1f90a80-4553-4d56-8c07-588390f7cba4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:47:15.547456 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f90a80-4553-4d56-8c07-588390f7cba4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:47:15.557413 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f90a80-4553-4d56-8c07-588390f7cba4 service placement] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:47:15.569345 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1455/2907] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:47:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:47:18.453960 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8270769-27fa-4519-bbc9-d7f662a26d4c service neutron] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:47:18.467274 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8270769-27fa-4519-bbc9-d7f662a26d4c service neutron] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:47:18.475939 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32fe170-8531-4365-ba57-036078f01df4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd503fe3cf6d428e962c14ab864ee608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:47:18.476562 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32fe170-8531-4365-ba57-036078f01df4 service placement] REQUEST_METHOD: `GET` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:47:18.476969 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32fe170-8531-4365-ba57-036078f01df4 service placement] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:47:18.477146 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32fe170-8531-4365-ba57-036078f01df4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:47:18.478964 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-d32fe170-8531-4365-ba57-036078f01df4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:47:18.493713 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32fe170-8531-4365-ba57-036078f01df4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:47:18.495661 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32fe170-8531-4365-ba57-036078f01df4 service placement] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:47:18.507386 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1453/2908] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:47:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:47:18.530876 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f90a80-4553-4d56-8c07-588390f7cba4 service placement] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:47:18.543774 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f90a80-4553-4d56-8c07-588390f7cba4 service placement] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:47:18.548760 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6611169-6f74-471c-b3e3-901bc99fb66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:47:18.549485 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6611169-6f74-471c-b3e3-901bc99fb66e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:47:18.549759 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6611169-6f74-471c-b3e3-901bc99fb66e admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:47:18.550169 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6611169-6f74-471c-b3e3-901bc99fb66e admin admin] PATH_INFO: `/v3/users/041280e59ffa4e518a599442fe9ded41` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:47:18.551614 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6611169-6f74-471c-b3e3-901bc99fb66e admin admin] RBAC: Authorizing `identity:delete_user(user_id=041280e59ffa4e518a599442fe9ded41)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:47:18.553753 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6611169-6f74-471c-b3e3-901bc99fb66e admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:47:18.610046 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a6611169-6f74-471c-b3e3-901bc99fb66e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '041280e59ffa4e518a599442fe9ded41'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:47:18.619634 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a6611169-6f74-471c-b3e3-901bc99fb66e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '041280e59ffa4e518a599442fe9ded41'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:47:18.633375 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a6611169-6f74-471c-b3e3-901bc99fb66e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '041280e59ffa4e518a599442fe9ded41'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:47:18.634384 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a6611169-6f74-471c-b3e3-901bc99fb66e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '041280e59ffa4e518a599442fe9ded41'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:47:18.642763 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-a6611169-6f74-471c-b3e3-901bc99fb66e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '041280e59ffa4e518a599442fe9ded41'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:47:18.649691 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1456/2909] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:47:18 2022] DELETE /identity/v3/users/041280e59ffa4e518a599442fe9ded41 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:47:18.835517 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32fe170-8531-4365-ba57-036078f01df4 service placement] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:47:18.846351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32fe170-8531-4365-ba57-036078f01df4 service placement] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:47:18.914357 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1bb07-dc70-4aa1-90a0-cc6b57abc24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:47:18.915060 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1bb07-dc70-4aa1-90a0-cc6b57abc24e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:47:18.915349 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1bb07-dc70-4aa1-90a0-cc6b57abc24e admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:47:18.915622 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1bb07-dc70-4aa1-90a0-cc6b57abc24e admin admin] PATH_INFO: `/v3/projects/65e721c90c8d41088ec9f9d92dd5858d` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:47:18.916805 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be1bb07-dc70-4aa1-90a0-cc6b57abc24e admin admin] RBAC: Authorizing `identity:delete_project(project_id=65e721c90c8d41088ec9f9d92dd5858d)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:47:18.918828 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be1bb07-dc70-4aa1-90a0-cc6b57abc24e admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:47:18.970650 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-6be1bb07-dc70-4aa1-90a0-cc6b57abc24e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65e721c90c8d41088ec9f9d92dd5858d', 'request_id': 'req-6be1bb07-dc70-4aa1-90a0-cc6b57abc24e'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:47:18.977664 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-6be1bb07-dc70-4aa1-90a0-cc6b57abc24e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65e721c90c8d41088ec9f9d92dd5858d', 'request_id': 'req-6be1bb07-dc70-4aa1-90a0-cc6b57abc24e'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:47:18.986772 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-6be1bb07-dc70-4aa1-90a0-cc6b57abc24e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65e721c90c8d41088ec9f9d92dd5858d', 'request_id': 'req-6be1bb07-dc70-4aa1-90a0-cc6b57abc24e'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:47:18.990895 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:47:19.013681 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-6be1bb07-dc70-4aa1-90a0-cc6b57abc24e admin admin] The token cache is being invalidate because project 65e721c90c8d41088ec9f9d92dd5858d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:47:19.013950 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-6be1bb07-dc70-4aa1-90a0-cc6b57abc24e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:47:19.014718 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-6be1bb07-dc70-4aa1-90a0-cc6b57abc24e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:47:19.016422 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1454/2910] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:47:18 2022] DELETE /identity/v3/projects/65e721c90c8d41088ec9f9d92dd5858d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:47:19.020783 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6611169-6f74-471c-b3e3-901bc99fb66e admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:47:19.022293 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-a6611169-6f74-471c-b3e3-901bc99fb66e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:47:19.036101 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6611169-6f74-471c-b3e3-901bc99fb66e admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:47:19.100235 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcf672d-8611-4a3b-8e7d-a0316d8acb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:47:19.100803 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf672d-8611-4a3b-8e7d-a0316d8acb9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:47:19.101083 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf672d-8611-4a3b-8e7d-a0316d8acb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:47:19.101354 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf672d-8611-4a3b-8e7d-a0316d8acb9e admin admin] PATH_INFO: `/v3/users/7ad61ef7d5454658a8d7d044c7441d56` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:47:19.102611 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcf672d-8611-4a3b-8e7d-a0316d8acb9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ad61ef7d5454658a8d7d044c7441d56)` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:47:19.104614 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcf672d-8611-4a3b-8e7d-a0316d8acb9e admin admin] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:47:19.156511 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-7dcf672d-8611-4a3b-8e7d-a0316d8acb9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ad61ef7d5454658a8d7d044c7441d56'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:47:19.159148 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:47:19.165132 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-7dcf672d-8611-4a3b-8e7d-a0316d8acb9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ad61ef7d5454658a8d7d044c7441d56'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:47:19.178519 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-7dcf672d-8611-4a3b-8e7d-a0316d8acb9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ad61ef7d5454658a8d7d044c7441d56'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:47:19.179217 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-7dcf672d-8611-4a3b-8e7d-a0316d8acb9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ad61ef7d5454658a8d7d044c7441d56'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:47:19.186065 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.notifications [None req-7dcf672d-8611-4a3b-8e7d-a0316d8acb9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ad61ef7d5454658a8d7d044c7441d56'} {{(pid=81934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:47:19.192013 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1457/2911] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Aug 23 04:47:19 2022] DELETE /identity/v3/users/7ad61ef7d5454658a8d7d044c7441d56 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:47:19.390069 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1bb07-dc70-4aa1-90a0-cc6b57abc24e admin admin] Authenticating user token {{(pid=81933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:47:19.391656 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.fernet_utils [None req-6be1bb07-dc70-4aa1-90a0-cc6b57abc24e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:47:19.405351 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1bb07-dc70-4aa1-90a0-cc6b57abc24e admin admin] Validating token access rules against request {{(pid=81933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:47:19.465536 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e0d7d1-1b2a-453f-a1cd-92dac093c680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de65eb39f13f4a92a6a526fb5ab1f5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba69f2ba4f0d496e9c7607a800699841', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:47:19.466133 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0d7d1-1b2a-453f-a1cd-92dac093c680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:47:19.466452 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0d7d1-1b2a-453f-a1cd-92dac093c680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:47:19.466720 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0d7d1-1b2a-453f-a1cd-92dac093c680 admin admin] PATH_INFO: `/v3/projects/c229879e107d4348ac054e1a7b5e8e95` {{(pid=81933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:47:19.468000 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e0d7d1-1b2a-453f-a1cd-92dac093c680 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c229879e107d4348ac054e1a7b5e8e95)` {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:47:19.469869 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e0d7d1-1b2a-453f-a1cd-92dac093c680 admin admin] RBAC: Authorization granted {{(pid=81933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:47:19.518195 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-97e0d7d1-1b2a-453f-a1cd-92dac093c680 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c229879e107d4348ac054e1a7b5e8e95', 'request_id': 'req-97e0d7d1-1b2a-453f-a1cd-92dac093c680'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:47:19.524626 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-97e0d7d1-1b2a-453f-a1cd-92dac093c680 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c229879e107d4348ac054e1a7b5e8e95', 'request_id': 'req-97e0d7d1-1b2a-453f-a1cd-92dac093c680'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:47:19.533672 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-97e0d7d1-1b2a-453f-a1cd-92dac093c680 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c229879e107d4348ac054e1a7b5e8e95', 'request_id': 'req-97e0d7d1-1b2a-453f-a1cd-92dac093c680'} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:47:19.544440 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-97e0d7d1-1b2a-453f-a1cd-92dac093c680 admin admin] The token cache is being invalidate because project c229879e107d4348ac054e1a7b5e8e95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 04:47:19.544728 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-97e0d7d1-1b2a-453f-a1cd-92dac093c680 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:47:19.545375 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG keystone.notifications [None req-97e0d7d1-1b2a-453f-a1cd-92dac093c680 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 04:47:19.546910 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: [pid: 81933|app: 0|req: 1455/2912] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Aug 23 04:47:19 2022] DELETE /identity/v3/projects/c229879e107d4348ac054e1a7b5e8e95 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 04:47:29.188538 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:DELETE=3,keystone:SELECT=12 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:47:29.541666 dv-cinder-854070-1-iscsi devstack@keystone.service[81933]: DEBUG dbcounter [-] [81933] Writing DB stats keystone:DELETE=8,keystone:SELECT=58,keystone:INSERT=1 {{(pid=81933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 04:51:38.961700 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcf672d-8611-4a3b-8e7d-a0316d8acb9e admin admin] Authenticating user token {{(pid=81934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 04:51:38.963947 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-7dcf672d-8611-4a3b-8e7d-a0316d8acb9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:51:39.044158 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcf672d-8611-4a3b-8e7d-a0316d8acb9e admin admin] Validating token access rules against request {{(pid=81934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 04:51:39.105387 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b19ff5b-b3e6-4bab-be3a-8d298ac8da29 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd503fe3cf6d428e962c14ab864ee608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f4373dfb5f9425eb73669e24d18a08f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 04:51:39.106006 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19ff5b-b3e6-4bab-be3a-8d298ac8da29 service placement] REQUEST_METHOD: `GET` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 04:51:39.106284 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19ff5b-b3e6-4bab-be3a-8d298ac8da29 service placement] SCRIPT_NAME: `/identity` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 04:51:39.106559 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19ff5b-b3e6-4bab-be3a-8d298ac8da29 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 04:51:39.108436 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.fernet_utils [None req-0b19ff5b-b3e6-4bab-be3a-8d298ac8da29 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 04:51:39.121090 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b19ff5b-b3e6-4bab-be3a-8d298ac8da29 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 04:51:39.122682 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b19ff5b-b3e6-4bab-be3a-8d298ac8da29 service placement] RBAC: Authorization granted {{(pid=81934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 04:51:39.135339 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: [pid: 81934|app: 0|req: 1458/2913] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Aug 23 04:51:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 04:51:49.129157 dv-cinder-854070-1-iscsi devstack@keystone.service[81934]: DEBUG dbcounter [-] [81934] Writing DB stats keystone:SELECT=58 {{(pid=81934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}